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FOREWORD 


Virtual  and  constructive  simulation  training  systems  have 
been  increasingly  used  by  the  Army  in  an  attempt  to  mitigate 
increasing  resource  constraints.  The  U.S.  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences  (ARI)  has  shown, 
however,  that  while  these  systems  can  be  very  effective,  the 
extent  of  this  effectiveness  depends,  in  part,  on  how  they  are 
used.  The  ARI's  Armored  Forces  Research  Unit  has  conducted 
research  and  development  efforts  into  structured  simulation- 
based  training  (SST)  in  order  to  maximize  the  Army's  ability  to 
realize  the  training  benefits  offered  by  its  simulation  training 
systems.  This  effort  has  been  directed  towards  collective 
training  requirements  of  combined  arms  forces  at  echelons  of 
brigade  and  below. 

A  companion  report  (Shlechter  &  Finley,  [in  preparation] ) 
describes  the  history  of  the  SST  programs,  and  analyzes  the  SST 
instructional  design  processes,  training  products,  and  lessons 
learned  as  documented  in  published  reports  and  training  support 
packages  (TSPs) .  The  purpose  was  to  make  widely  available  the 
knowledge  and  insights  gained  from  the  SST  programs.  This 
report  completes  the  effort.  It  draws  upon  interviews  and 
questionnaires  completed  by  key  SST  developers,  providing 
important  insights  beyond  those  originally  documented  and 
illuminating  actions  necessary  to  make  the  use  of  SST  TSPs  a 
routine  part  of  the  Army's  training  practices. 

This  report  was  completed  under  the  Armored  Forces  Research 
Unit's  Work  Package  205,  Assessment  of  Force  XXI  Training  Tools 
and  Techniques  (AFT3).  The  reviewed  SST  efforts  were  performed 
under  two  work  packages:  Strategies  for  Training  and  Assessing 
Armor  Commanders '  Performance  with  Devices  and  Simulations 
(STRONGARM)  and  Force  XXI  Training  Methods  and  Strategies 
(FASTTRAIN) .  The  research  was  completed  pursuant  to  a 
Memorandiim  of  Agreement  with  the  U.S.  Army  Armor  Center  and  Fort 
Knox:  Manpower,  Personnel  and  Training  Research,  Development, 

Test,  and  Evaluation  for  the  Mounted  Forces,  16  October  1995. 

The  information  in  this  report  has  been  provided  to 
training  developers  and  instructors  at  the  U.S.  Army's  Armor 
School  at  Fort  Knox.  This  report  also  has  ramifications  for  the 
development  and  implementation  of  future  SST  programs . 

M.  SIMUTIS 
lical  Director 
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COMBINED  ARMS  STRUCTURED  SIMULATION-BASED  TRAINING  PROGRAMS: 
REFLECTIONS  OF  KEY  DEVELOPERS 


executive  summary 


Research  Requirement : 

The  Armored  Forces  Research  Unit  of  the  U.S.  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences  ^RI  completed 
a  series  of  structured  simulation-based  training  (SST)  ^esearc 
and  development  (R&D)  programs  during  the  period,  1993  1999. 

These  programs  focused  on  developing  SST  training  support 
packagL  (TSPs)  to  meet  collective  training  requirements  for  the 

combined  arms  at  echelons  of  brigade  and  below.  An  SST 
development  model /methodology  had  also  been  produced  for 
addressing  additional  training  needs  as  required.  A  companio 
report  to  this  one  (Shlechter  &  Finley,  [m  preparation] ) 
provides  an  integrated  history  of  these  programs,  and  accrued 
lessons  learned  that  can  be  applied  to  future  SST  TS 
development  efforts.  In  preparation  for  writing  that  report, 
the  published  SST  reports  and  their  TSP  products  were  reviewed 
and  analyzed,  and  discussed  with  others.  Areas  requiring 
further  clarification  and/or  investigation  were  identified. 

TheL  were:  instructional  design,  remaining  research  needs,  and 

ImplemLtation  and  fielding.  This  report  addresses  these  three 

areas . 

Procedure : 

New  and  additional  information  was  needed  to  address  the 
three  areas.  Principal  sources  for  this  information  were  key 
developers  who  had  participated  in  ARI's  SST  R&D  programs . 

These  persons  included  ARI's  Contracting  Officer  s 
Representatives  (CORs)  and  persons  selected  from  the  consortia 
of  SST  contractors.  Upon  request,  these  persons  responded  to 
structured  interviews  and  questionnaires.  These  instruments 
were  designed  to  gain  information  related  to:  personal 
backgrounds;  experiences  in  the  completed  SST  programs;  and 
reflections  and  subsequent  observations  providing  new  insig 
and  information. 

Findings : 

The  findings  addressed  each  of  the  three  areas  with  respect 
to:  the  respective  roles  of  the  constructivism  and  ^ 

behaviorism/ Systems  Approach  to  Training  (SAT)  instructiona 
theories;  specific  needs  for  further  information  and  research, 
and  planning  and  logistical  requirements  for  integrating  and 
maintaining  SST  as  a  part  of  the  U.S.  Army  training  system. 

The  relative  use  of  the  two  instructional  theories  was 
examined  across  the  SST  R&D  programs.  The  SST  programs 
along  two  major  dimensions:  the  echelons  and  the  types  o 
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beina  trained.  It  was  found  that  selections  of  elements  from 
each  of  the  two  theories  varied  in  a  consistent  manner  across 
ARI ' s  SST  programs,  even  though  behaviorism/ SAT  elements 
predominated  in  every  case.  The  change  was  a  transition  towards 
greater  use  of  constructivism  in  those  SST  programs  focusing  o 
^h^higher  echelons,  and  on  coiranand  and  battle  staff  (as  opposed 
to  coirtoat)  types  of  tasks.  The  choices  of  design  elements  by 
the  key  developers  were  found  to  have  been  based  primarily  on 
examination  of  the  specifics  of  each  training  requirement.  The 
results  were  hybrid  applications  of  the  two  theories  in  every 

case . 

Needs  for  front  end  analyses  (FEAs) ,  research,  and 
enhancements  to  the  SST  development  model  were  well  specified. 
The  FEAs  were  seen  as  an  effective  way  to  gain  needed 
information.  The  information  would  especially  be  used  as  a 
basis  for  determining  how  to  best  integrate  SST  into  the  U.S. 
Army  training  system  and  as  a  means  for  evaluating  possibilities 
for^more  efficiLt  training.  The  needs  identified  for  research 
primarily  concerned  transfer  of  training  issues.  These  issues 
related  not  only  to  SST,  but  also  to  simulation-based  training 
in  general.  A  singular  effort  was  suggested  to  enhance  the  SST 
development  model,  one  to  improve  those  software  capabilities 
needed  to  implement  the  structures  designed  for  the  TSP 
scenarios  and  subsequent  modifications  to  them. 

Finally,  based  on  comments  regarding  the  considerable 
potential  for  SST  benefits  and  concerning  needs  for  sustainment 
and  support  if  these  benefits  are  to  be  realized,  certain 
actions  were  proposed.  These  actions  are  intended  to  satis  y 
SST  sustainment  and  support  requirements  in  a  manner  assuring 
realization  of  the  SST  training  benefits.  These  proposed 
actions  include  development,  enactment,  and  implementation  of: 
an  Arrry  SST  Master  Plan,  an  SST  logistics  system,  and  attendant 

policies . 

Utilization  of  Findings: 

The  companion  report  (Shlechter  &  Finley,  [in  preparation]) 
that  stimulated  this  investigation  has  ramifications  for  both 
the  military  training  and  instructional  design  communities. 

Both  communities  should  be  able  to  use  that  report  as  a  building 
block  for  future  SST  efforts.  This  report  provides  new 
information  that  complements  the  companion  report  for  ooth 
communities.  In  addition,  research  issues  are  discussed  that 
would  interest  instructional  theorists  and  researchers.  Lastly, 
this  report  offers  recommendations  for  actions  that  should 
esoecially  benefit  military  training  implementers ,  users,  and 
managers,  and  those  logisticians  with  training-related  concerns. 
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COMBINED  ARMS  STRUCTURED  SIMULATION-BASED  TRAINING  PROGRAMS: 

REFLECTIONS  OF  KEY  DEVELOPERS 


INTRODUCTION 

This  report  represents  the  second  portion  of  a  two-part 
examination  of  issues  related  to  structured  simulation-based 
training  (SST) .  A  companion  report  (Shlechter  &  Finley,  [in 
preparation] )  provides  a  detailed  description  of  research  and 
development  (R&D)  programs  executed  to  produce  SST  for  the  U.S. 
Army.  These  programs  focused  on  collective  training 
requirements  for  the  combined  arms  at  echelons  of  brigade  and 
below;  all  were  executed  under  the  direction  of  the  Armored 
Forces  Research  Unit,  located  at  Fort  Knox,  KY,  an  element  of 
the  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social 
Sciences  (ARI) . 

The  companion  report  focuses  on  the  documented  SST  R&D 
history,  products  (i.e.,  training  support  packages,  or  TSPs) , 
and  lessons  learned.  Based  on  review  and  analyses  of  this 
documentation,  a  total  of  43  lessons  learned  were  identified. 
These  lessons  were  intended  to  support  future  efforts  initiated 
to  develop  SST  TSPs.  One  such  lesson  learned,  especially 
relevant  to  this  report,  was  that  future  SST  efforts  should 
probably  take  a  hybrid  approach  to  the  application  of 
instructional  design  concepts. 

The  companion  report  also  provides  an  integrated  overview 
of  ARI's  SST  R&D  programs.  The  integration  encompassed  a  very 
large  nuit±)er  of  very  substantial  documents.  The  report  should 
therefore  provide  not  only  an  overall  perspective,  but  also 
serve  as  a  guide  for  those  wishing  to  determine  which  documents 
would  best  serve  them  when  addressing  more  detailed  questions. 

Based  on  reviews  and  analyses  of  the  documentation,  it 
appeared  that  one  or  more  issues  could  benefit  from  a  closer 
examination.  For  example,  even  though  the  use  of  a  hybrid  SST 
design  approach  was  apparent  based  on  the  documentation,  the 
reasons  for  a  hybrid  approach  or  the  exact  constitution  of  that 
approach  were  not  clear.  Moreover,  discussions  participated  in 
while  preparing  the  companion  report  suggested  that  some 
valuable  new  insights  may  have  evolved  based  on  retrospection 
and  subsequent  observations.  A  second  report  on  SST  appeared  to 
be  warranted.  Emerging  topic  areas  appeared  to  be  instructional 
design,  SST  implementation  and  fielding,  and  remaining  SST 
research  needs . 

Given  the  foregoing,  efforts  to  collect  additional 
information  were  initiated  and  this  report  is  the  result.  The 
principal  sources  for  this  information  were  those  personnel 
directly  involved  in  ARI's  SST  R&D  programs.  Ones  identified  as 
especially  key  persons  did,  upon  request,  provide  the  necessary 
information.  These  key  program  participants  included  ARI's 
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Contracting  Officer's  Representatives  (CORs)  and  those  persons 
they  selected  from  their  consortia  of  SST  contractors.  The  CORs 
and  selected  contractors  are  referred  to  throughout  this  report 
as  "key  developers."  They  provided  the  information  sought  by 
responding  to  questionnaires  and  structured  interviews. 

This  report  describes  the  findings  and  the  collected  data 
found  to  be  most  relevant.  The  findings  cluster  into  three 
major  areas.  These  areas  are: 

♦  Contributions  of  the  constructivism  and  behaviorism 
instructional  theories  to  the  SST  programs; 

♦  Needs  for  further  information  and  research;  and 

♦  Planning  and  logistical  requirements  for  integrating  SST 
into  the  Army's  training  system. 

The  likely  target  audiences  for  these  three  areas  are:  training 
and  instructional  developers,  researchers,  and  theorists; 
training  implementors,  users,  and  managers;  and  logisticians 
with  training-related  responsibilities. 

The  report  is  divided  into  six  sections.  First,  a 
Background  section  presents  a  brief  history  of  the  SST  programs, 
a  description  of  the  targeted  training  participants  and  the  key 
developers,  and  the  rationales  and  issues  guiding  preparation  of 
this  report.  The  next  section  describes  the  methods  used  to 
collect  and  analyze  the  data.  The  findings,  or  Results,  are 
presented  in  three  sections  aligned  with  the  areas  listed  above. 
The  report  closes  with  a  Summary  and  Conclusions  section. 

BACKGROUND 

A  Brief  History  of  the  SST  R&D  Programs 
Need  for  SST  R&D 


Simulations  as  a  training  capability.  During  the  last 
decade,  there  has  been  a  reduction  in  the  U.S.  Army's  fiscal 
allocations  for  such  personnel  as  instructors,  unit  trainers, 
and  training  developers  (Department  of  the  Army  [DA],  1999). 
There  has  also  been  a  sharp  reduction  in  fiscal  allocations  for 
training  supplies  such  as  ammunition  and  fuel.  Training 
resources  have  always  been  limited.  They  have  become  even  more 
limited  in  recent  years. 

To  meet  this  resource  challenge,  the  U.S.  military  has 
increasingly  relied  on  virtual  and  constructive  simulation-based 
systems  to  provide  training  to  its  combat  forces.  These 
simulation  systems  are  less  resource-intensive  than  traditional 
military  field  exercises,  or  "live"  simulations.  Virtual 
systems  involve  immersing  the  training  audience  into  tactical 
situations  where  direct  contact  with  conditions  on  the 
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battlefield  itself  are  closely  approximated  (e.g. ,  sitting  in  a 
simulated  tank  while  viewing  simulated  terrain  and  enemy  tanks) . 
Constructive  simulations  provide  training  participants  with 
tactical  scenarios  through  use  of  computer-driven  models  of  the 
battlefield.  Here,  the  interfaces  with  the  battlefield  are  more 
indirect  and  much  like  what  would  be  experienced  in  a  command 
post  (e.g.,  computer-driven  displays  of  changing  data  and  of 
icons  representing  enemy  units  on  maps) .  Over  the  years,  many 
virtual  and  constructive  simulation  systems  have  been 
demonstrated  to  be  capable  of  training  effectively  (e.g., 

Finley,  Rheinlander,  Sullivan,  &  Thompson,  1971;  Bessemer, 

1991)  . 

The  virtual  systems  for  the  SST  programs  are  the  Simulation 
Network  (SIMNET)  and  the  Close  Combat  Tactical  Trainer  (CCTT) 
systems.  The  constructive  systems  are  the  Janus  and 
Brigade/Battalion  Battle  Simulation  (BBS)  systems.  Another 
system  used  in  the  SST  programs,  the  Staff  Group  Trainer  (SGT) , 
can  be  described  as  a  hybrid  virtual-constructive  system.  A 
factor  differentiating  these  systems  is  the  extent  to  which  they 
provide  the  types  of  realism  found  in  a  real-world  situation. 
This  factor  depends  on  the  number  of  real-life  variables  that 
can  be  simulated  with  apparent  face-validity.  The  CCTT  is  able 
to  replicate  more  variables  than  the  older  SIMNET.  The  same  is 
true  for  the  BBS  when  compared  to  Janus.  (In  both  cases,  there 
is  also  a  differentiating  factor  of  replication  scope,  but  this 
factor  is  not  a  major  concern  for  this  report.) 

A  structured  approach  to  simulation-based  training:  A 
definition.  An  approach  of  structuring  training  such  that  it 
more  or  less  focuses  on  achievement  of  specified  training 
objectives  has  often  been  acknowledged  as  essential  to  the 
achievement  of  efficient  and  effective  training  (e.g..  Joint 
Simulation  System  (JSIMS)  Learning  Methodology  Working  Group, 
1999) .  In  developing  structured  strategies  tailored  to  the 
environment  afforded  by  simulation-based  training  systems,  ARI's 
SST  R&D  programs  have  defined  the  key  elements  to  be  (Campbell, 
Campbell,  Sanders,  &  Flynn,  1995;  Campbell  &  Deter,  1997): 

♦  Training  exercises  are  designed  to  embed  specified 
training  objectives  into  a  predetermined,  well  detailed, 
scenario . 

♦  That  scenario  is  designed  to  provide  cues,  or  triggering 
events,  for  the  training  participants. 

♦  These  cues  are  intended  to  cause  the  participants  to 
perform  the  desired  behaviors  and  then  experience  the 
consequences  of  their  actions. 

Problem:  Failure  to  sufficiently  realize  potential 

training  benefits.  Many  simulation  systems  have  been  fielded 
without  standardized  training  programs.  Unfortunately,  even 
where  standardized  programs  have  been  made  available,  many  of 
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these  have  not  been  intentionally  structured  as  needed  to  meet 
well-specified  training  requirements.  The  fielding  of  SIMNET 
was  one  example  of  no  standardized  training  programs  being 
available,  let  alone  structured.  Here,  decisions  regarding 
which  SIMNET  battlefield  environment  to  request,  which  mission 
to  execute,  and  specification  of  the  scenario  details  were 
considered  to  be  the  responsibility  of  the  unit  requesting  the 
training.  Hence,  realization  of  SIMNET' s  training  potential 
depended  on  qualifications  of  the  unit's  personnel:  their 
knowledge  of  SIMNET 's  simulation  capabilities  and  limitations, 
their  having  adequately  specified  their  training  needs 
beforehand,  and  their  understanding  of  how  to  relate  SIMNET' s 
capabilities  to  their  needs.  The  SIMNET  site  staff  would  assist 
the  unit,  but  this  was  difficult  when  the  unit's  training 
objectives  often  were  not  well  articulated.  Under  these 
conditions,  exercises  often  evolved  into  ones  that  were  largely 
free-play  and,  upon  completion,  it  was  not  clear  what,  if  any, 
training  needs  had  been  met . 

To  the  extent  a  unit  was  able,  however,  to  specify  its 
requirements  and  did  so  within  the  perceived  SIMNET  environment, 
the  unit  imposed  at  least  some  structure  on  their  simulation- 
based  training;  that  is,  they  were  accomplishing  SST  to  some 
degree  at  least.  A  corollary  to  the  foregoing  observation  is 
that  such  a  unit  would  also  avoid  using  simulation  systems  like 
SIMNET  in  a  completely  free-play  mode  because  they  would 
understand,  whether  or  not  explicitly,  that  free-play  was  the 
antithesis  of  the  structure  they  needed  to  achieve  their 
training  objectives. 

However,  the  expertise  found  to  exist  in  units  regarding 
how  to  specify  training  needs  and  then  relate  these  to 
simulation  system  capabilities  is  generally  limited.  It  is  not 
surprising  that  Bessemer  (1991)  determined  that  the  training 
needs  met  and  the  effectiveness  of  SIMNET  in  meeting  these  needs 
varied  from  training  unit  to  training  unit.  These  variations 
were  found  to  be  due  largely  to  the  foregoing  limitations. 
Another  constraint  in  exploiting  Army  simulation-based  training 
systems,  in  addition  to  not  always  using  simulation  systems  to 
their  best  advantage,  was  that  many  units  spent  considerable 
time  specifying  their  exercises  at  training  sites  after  they 
arrived  -  rather  than  using  their  precious  and  limited  time  to 
actually  train  (Shlechter,  Bessemer,  Nesselroade,  &  Anthony, 
1995)  . 

SST  R&D  in  response  to  the  problem.  In  light  of  the 
foregoing,  the  ARI  SST  programs  were  R&D  efforts  directed  toward 
enabling  the  Army  to  better  exploit  their  simulation-based 
training  systems.  They  involved  the  development  of  procedures 
to  create  standardized  structured  training  exercises,  each 
serving  a  particular  mix  of  clearly  identified  training 
requirements.  In  so  doing,  ARI  also  created  some  initial  sets 
of  these  SST  exercises  (i.e.,  TSPs) .  Given  the  availability  of 
these  standardized  SST  exercises,  units  could  then  select  those 
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which  best  met  their  training  needs.  Further,  units  could  do 
this  prior  to  their  arrival,  thus  freeing  up  time  for  training 
at  the  simulation  system's  site. 

The  particular  set  of  training  requirement (s)  addressed  by 
each  individual  TSP  produced  under  ARI's  SST  program  was 
determined  largely  by  the  primary  focus  of  each  SST  project.  An 
overall  description  of  the  training  requirement (s)  and 
audience (s)  addressed  across  the  SST  TSPs  is  provided  by 
considering  the  five  factors  listed  below.  The  general 
requirements  addressed  by  a  specific  SST  TSP  can  be  described  by 
selecting,  as  appropriate,  one  or  more  of  the  options  listed 
with  each  of  the  factors: 

♦  Task  areas :  Combat,  and  Command  and  Battle  Staff; 

♦  Missions ;  Movement  to  Contact,  Defense,  and  Deliberate 
Attack; 

♦  Mission  phases:  Plan,  Prepare,  and  Execute  (in  some 
cases,  the  Execution  elements  of  consolidation  and 
reorganization  were  included) ; 

♦  Echelons :  Platoon,  Company /Troop,  Battalion,  and 
Brigade /Brigade  Combat  Team;  and 

♦  Branches :  Armor,  Cavalry,  Infantry,  and  those  branches 
appropriate  for  selected  Combat  Support  (CS)  and  Combat 
Service  Support  (CSS)  roles  (e.g..  Signal,  Engineer). 

A  Snapshot  of  the  SST  Projects 


A  total  of  ten  projects  were  performed  under  ARI's  SST  R&D 
program.  These  ten  projects  will  be  considered  in  this  report 
to  group  into  four  sets  of  projects.  These  four  sets  will 
generally  be  designated  as  "programs,"  which,  together, 
constitute  the  overall  ARI  SST  R&D  program.  The  four  sets  of 
projects  are: 

♦  The  Virtual  Training  Program  (VTP) ,  which  was  developed 
during  fiscal  years  1993  through  1996.  For  the  purposes 
of  this  report,  VTP  includes  three  projects:  Simulation- 
based  Multi-echelon  Training  Program  for  Armor  Units 
(SIMUTA) ,  SIMUTA-Battalion  Exercise  Expansion  (SIMUTA-B) , 
and  Simulation-Based  Mounted  Brigade  Training  Program 
(SIMBART) . 

♦  Structured  Training  for  the  Close  Combat  Tactical  Trainer 
(STRUCCTT) ,  which  occurred  during  fiscal  years  1997 
through  1999. 

♦  Staff  Group  Trainer  (SGT) ,  which  transpired  during  fiscal 
years  1997  through  1998. 
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♦  Combined  Arms  Operations  at  Brigade  Level,  Realistically 
Achieved  through  Simulation  (COBRAS) ,  which  took  place 
during  fiscal  years  1995  through  1998. 

Special  designated  congressional  funds  were  used  to  initiate  the 
SIMUTA  and  COBRAS  R&D  efforts. 

The  "VTP"  designation  covers  all  SIMUTA  and  SIMBART  efforts 
in  this  report  because  of  the  manner  in  which  questionnaire  and 
interview  items  were  constructed  and  then  discussed  with  the  key 
developers.  Where  STRUCCTT  is  the  sole  topic  or  is  a  referent 
in  a  discussion  along  with  VTP,  the  respective  designations  of 
either  STRUCCTT  or  VTP /STRUCCTT  will  be  used. 

Figures  la  and  lb  describe  the  alignment  of  the  four 
programs  with  the  five  factors  listed  above  that  can  be  used  to 
characterize  the  many  training  requirements  addressed:  tasks, 
missions,  mission  phases,  echelons,  and  branches.  The  programs 
are  aligned  with  their  approximate  program  execution  timelines. 
If  further  details  are  desired,  see  Shlechter  and  Finley  (in 
preparation) . 

Targeted  Training  Participants 

The  targeted  training  participants  for  these  R&D  efforts 
were  combined  arms  soldiers .  The  central  focus  was  on  the 
combat  arms  branches  of  Armor,  Cavalry,  and  (mechanized) 
Infantry.  Specific  training  objectives  for  selected  CS  and  CSS 
functions  were  introduced  in  the  SGT  and  COBRAS  programs .  This 
led  to  members  of  branches  appropriate  to  those  roles  (e.g.. 
Signal,  Engineer)  becoming  major  training  participants  as  well, 
rather  than  acting  only  as  role  players  when  necessary. 

The  "Key  Developers" 


Many  military,  civilian  government,  and  civilian  contractor 
personnel  participated  in  the  SST  R&D  efforts.  Persons  from  the 
latter  two  groups  constituted  the  "key  developers"  who 
contributed  the  "reflections"  documented  in  this  report. 

Civilian  government  group.  The  contributing  members  of  the 
civilian  government  group  were  the  two  SST  R&D  Contracting 
Officer's  Representatives  (CORs) .  These  CORs  formulated  and 
designed  the  programs,  and  were  responsible  for  both  guiding  the 
programs  technically  and  monitoring  performance  from  a 
contractual  standpoint.  They  provided  interpretations  of  the 
Army's  interests,  needs,  and  circumstances  to  the  contractors, 
as  well  as  their  own  technical  expertise.  Familiarity  with  the 
problem  area  included  their  experience  as  Army  officers  and  as 
CORs  for  other  training  R&D  efforts. 
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Figure  la.  Approximate  time  line  of  the  VTP  and  STRUCCTT  sets  of  SST  programs  in 
response  to  maneuver  unit  collective  training  requirements. 


STRUCTURED  SIMULATION-BASED  TRAINING  PROGRAMS 


D) 

a 

•H 

•H 

03 

U 

4J 


o3 

G 

O 

-H 

•H 

cn 

a 

fd 


t3 

a 


(D 

tn 

0) 


Figure  lb.  Approximate  time  line  of  the  SGT  and  COBRAS  sets  of  SST  programs  in  response  to 
maneuver  unit  collective  training  requirements. 


civilian  contractor  group.  The  civilian  contractors 
molded,  developed,  and  implemented  (during  initial  tryouts)  the 
SST  TSPs  in  response  to  the  CORs '  guidance.  The  CORs  selected 
15  of  these  persons  to  represent  the  consortia  of  SST 
contractors  for  the  present  information  collection  effort. 

These  persons,  as  a  group,  provided  a  representative  sample  of 
experience  across  the  SST  programs  and  were  persons  who  also 
served  in  especially  important  roles.  This  contractor  group  was 
comprised  of  SST  program  leaders  and  technical  personnel  having 
expertise  as  instructional  designers,  evaluators,  technicians, 
and  users,  as  well  as  expert  knowledge  regarding  military 
perspectives .  The  contractors  selected  were  ones  who  had 
experienced  an  intense  program  involvement  of  relatively  long 
duration.  Many  of  them  also  had  experience  in  more  than  one 
role  and  on  more  than  one  program. 

More  detailed  information  was  obtained  on  the  contractors' 
prior  experience,  and  their  roles  and  responsibilities  on  the 
SST  programs,  through  use  of  a  questionnaire  (see  Appendix  A, 
Part  A) .  While  the  questionnaire  itself  will  be  described  in 
the  Method  section,  the  data  obtained  on  selected  items  are 
presented  below.  The  purpose  is  to  provide  the  reader  with  a 
quantitative  understanding  of  the  composition  of  the  contractor 
group  in  terms  of  types  of  prior  experience,  and  roles  and 
experiences  in  the  different  SST  programs  and  projects.  This 
information  is  presented  now  to  avoid  distracting  later 
discussions  of  findings  from  the  points  being  made. 

These  background  data  provide  a  picture  of  the  contractor 
key  developers  particularly  useful  for  appreciating  the 
knowledge  base  from  which  they  spoke.  Responses  described  the 
contractors'  training-related  experiences  prior  to  working  on 
ARI's  SST  programs,  number  of  years  involved  in  ARI's  programs, 
and  the  ARI  SST  programs  on  which  they  worked.  These  responses 
are  tabulated  in  Table  1.  With  regard  to  prior  experience,  all 
participants,  with  the  exception  of  one,  indicated  having  had 
three  or  more  of  the  experiences  listed  in  Table  1.  All 
indicated  that  they  had  worked  on  ARI's  SST  programs  for  at 
least  two  years,  while  60%  indicated  they  had  worked  on  them  for 
five  or  more  years. 

Data  presented  in  Table  1  describing  participation  in  the 
SST  programs  makes  it  clear  that  many  of  the  contractors  worked 
on  more  than  one  program  (12+7+8  does  not  equal  15) .  To 
clarify  the  picture  of  involvements  in  SST  programs,  a  Venn 
diagram  depiction  is  provided  in  Figure  2 .  It  can  be  seen  from 
this  figure  that  only  four  contractors  worked  on  only  one  group 
of  SST  programs.  Of  these  four,  two  worked  exclusively  on 
VTP/STRUCCTT,  while  the  other  two  worked  exclusively  on  COBRAS. 
It  can  also  be  seen  that  only  one  contractor  worked  on  all  three 
program  groups . 
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Table  1 .  Data  describing  the  contractors '  prior  experiences  and 
involvement  in  the  SST  R&D  programs . 


BACKGROUND  AND  PROGRAM  INVOLVEMENT 
OF  THE  CONTRACTOR  KEY  DEVELOPERS 
(QUESTIONNAIRE  ITEMS  1-3) 

FREQUENCIES  & 
PROPORTIONS 

N  =  15 

HAD  THESE  PRIOR  RELATED  EXPERIENCES  (Ql) 

Student /trainee  in  a  military 
instructional  program 

10  (.67) 

Member  of  a  military 
instructional  team 

12  (.80) 

Instructional  designer 

10  (.67) 

Member  of  a  program's 
evaluation  team 

8  (.53) 

Student/ trainee  in  a  civilian 
instructional  program 

10  (.67) 

Something  other  than  the 
roles  described  above 

4  (.27) 

None  of  the  above  -  this  is 
my  first  experience 

1  (.07) 

INVOLVED  IN  THE  ARI  SST  R&D  FOR  HOW  LONG  (Q2) 

Five  or  more  years 

9  (.60) 

Two  -  four  years 

6  (.40) 

One  year  or  less 

PARTICIPATED  IN  THESE  SST  R&D  PROGRAMS  (Q3) 

VTP/STRUCCTT 

12  (.80) 

SGT 

7  (.47) 

COBRAS 

8  (.53) 

Rationales  and  Issues  Guiding  this  Report 


As  indicated  in  the  Introduction,  the  information  obtained 
for  this  report  fell  into  three  areas:  the  contributions  of  the 
constructivism  and  behaviorism  instructional  theories  to  the  SST 
programs;  needs  for  further  information  and  research;  and 
planning  and  logistics  requirements  for  SST.  As  the  importance 
of  the  second  and  third  areas  should  be  obvious,  no  underlying 
rationales  and  issues  will  be  discussed  here.  In  contrast, 
while  issues  surrounding  alternative  instructional  theories  and 
development  of  many  types  of  instructional  programs,  including 
SST  TSPs,  are  important,  they  are  not  always  appreciated  or 
addressed  as  a  part  of  development.  Therefore,  discussion  of 
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Figure  2 .  SST  program  experiences  of  the  15  questionnaire 
respondents . 


the  background  and  rationale  for  addressing  this  issue  is 
presented  here. 

During  the  period  of  the  SST  R&D,  1993-1999,  a  newer 
instructional  theory,  called  constructivism,  was  gaining  the 
spotlight  and  being  compared  to  traditional  behavioristic 
approaches  (cf..  The  Cognition  and  Technology  Group  at 
Vanderbilt,  1992;  Duffy  &  Jonassen,  1992;  Willis,  1995). 
Behavioristic  approaches  were  best  known  to  the  military,  as 
well  as  in  many  other  circles,  as  those  procedures  constituting 
the  Systems  Approach  to  Training  and  Instructional  Systems 
Development  (SAT,  ISD,  or  SAT/ISD) .  The  labels,  SAT  and  ISD, 
have  been  institutionalized  through  documentation  (e.g., 
Branson,  1978;  Branson  &  Grow,  1987),  common  use,  and 
regulations  governing  their  use  (e.g..  Department  of  the  Air 
Force,  1970;  DA,  1995;  Department  of  the  Navy,  1981) .  Major 
differences  between  the  approaches  of  constructivism  and 
behaviorism  are  described  in  Table  2  with  regard  to 
instructional  emphases. 
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Table  2.  Instructional  emphases 
behaviorism  theories . 

of  the  constructivism  and 

INSTRUCTIONAL  EMPHASES 

CONSTRUCTIVISM 

BEHAVIORISM 

Instructional  objectives  emerge 
from  decisions  made  by  the 
training  participants. 

Instructional  objectives  are 
determined  by  analysts  before 
training  is  provided. 

Authentic  instructional 
conditions . 

Instructional  objectives  are 
the  fulcra  for  developing  the 
instructional  materials. 

Training  participants  develop 
an  understanding  of  the 
principles  of  tactics. 

Training  participants  learn  how 
to  execute  tactical  actions. 

After  action  reviews  (AARs)  are 
led  by  the  students. 

AARs  are  led  by  the 
instructors . 

Based  on  evidence  dravm  from  analyses  of  the  TSPs  and 
discussions  with  the  CORs,  Shlechter  and  Finley  (in  preparation) 
observed  that  "SST  programs  should  seemingly  be  a  hybrid  of  the 
behavioral  and  constructivist  approaches  to  instructional 
design."  (p.  99).  Shlechter  and  Finley  were  not,  however,  able 
to  delineate  the  contributions  of  the  two  approaches  to  the 
degree  desired. 

The  published  SST  reports  generally  described  procedural 
aspects  of  the  SST  TSP  development  and  instructional  methods, 
often  to  a  fairly  detailed  level.  This  was  apropos,  given  the 
R&D  goal  to  provide  the  U.S.  Army  with  not  only  a  working  sample 
of  TSPs,  but  also  a  methodology  for  developing  additional  ones 
in  the  future.  The  reports  did  not,  however,  provide  explicit 
discussions  of  alternative  instructional  theories  and  approaches 
shaping  these  procedures.  Rather,  the  reports  pointed  to  SAT  as 
being  their  basis  and  then  described  those  procedural  changes 
made  to  SAT  when  needed  to  better  address  changes  taking  place 
in  the  instructional  requirements.  (These  changes  in 
requirements  resulted  from  changes  in  the  training  audiences 
from  combat  units  at  platoon  and  company  echelons  to  command  and 
battle  staffs  at  battalion  and  brigade  echelons  [see  Figures  la 
and  lb] . ) 
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From  the  standpoint  of  instructional  theory  and  design,  the 
changes  appeared  to  reflect  at  least  some  movement  from 
traditional  behaviorism,  or  SAT/ISD,  to  the  newer  constructivism 
approach.  However,  lacking  articulation  in  the  reports  of  the 
analysis  and  methodological  changes  from  the  perspective  of 
SAT/behaviorism  and  constructivism  features,  it  was  not  clear 
whether  these  alternative  instructional  design  approaches  had 
been  explicitly  examined  or  not. 

From  ARI's  perspectives,  the  research  and  utilitarian  ones, 
the  application  of  alternative  instructional  theories  to  SST  and 
the  reasons  for  choosing  one  theory  as  compared  to  the  other  are 
important.  On  occasion,  past  comparisons  between  constructivism 
and  SAT/ISD  approaches  have  been  presented  as  oppositional 
rather  than  addressing  how  to  make  the  best  selections  of  their 
respective  features.  We  suspect  that  an  oppositional  view  may 
not  always  be  in  the  best  interests  of  meeting  training  needs. 

It  may  rather  be  the  case  that  a  better  course  of  action  would 
be  to  consider  alternative  instructional  concepts  and  to  do  so 
in  terms  of  any  special  aspects  of  the  training  requirements. 

At  least  one  effort  is  known  which  did  take  this  approach. 
That  effort  explicitly  examined  alternative  instructional 
theories,  including  constructivism  and  behaviorism,  to  identify 
features  best  suited  to  a  particular  training  need  associated 
with  the  U.S.  Field  Artillery  Officer  Advanced  Course  (FAOAC) 
(Ross  &  Pierce,  draft  1998) .  The  authors  described  their 
results  as  a  framework,  or  strategy,  which  drew  from  features  of 
each  theory  as  was  deemed  most  appropriate  for  meeting  the 
specific  instructional  requirements  and  conditions  of  FAOAC. 

The  ARI  SST  R&D  program  offers  an  opportunity  to  move  a 
step  beyond  Ross  and  Pierce's  work  (draft  1998),  where  only  one 
course  of  instruction  was  examined.  The  overall  ARI  program  can 
be  separated  into  two,  three,  or  four  programs,  depending  on 
which  distinctions  are  useful  for  the  discussion  at  hand:  VTP/ 
STRUCCTT  and  SGT/COBRAS.  As  there  are  some  major  differences  in 
the  training  requirements  addressed  across  these  SST  programs, 
differences  found  between  them  with  regard  to  instructional 
design  could  be  of  interest.  If  any  of  these  differences  could 
be  described  as  reflecting  choices  between  constructivism  and 
behaviorism,  this  would  be  of  interest  from  a  research 
standpoint.  From  the  standpoint  of  instructional  development, 
this  could  serve  utilitarian  purposes  as  well. 

METHOD 

Research  Approach 

The  approach  consisted  of  obtaining  information  from  the 
two  groups  constituting  the  key  developers:  the  SST  R&D  CORs  (N 
=  2)  and  selected  contractor  personnel  (N  =  15).  These  17 
persons  were  described  in  the  Background  section  and  noted  to  be 
the  "key  developers"  referenced  in  the  title  of  this  report. 
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Both  questionnaire  and  interview  formats  were  used  and  are 
described  below.  The  same  interview  team  of  two  persons 
conducted  all  data  collection  sessions. 

Only  interview  data  were  obtained  from  the  CORs .  Both 
interview  and  questionnaire  data  were  obtained  from  the 
contractors.  The  contractors,  on  an  individual  basis,  each 
completed  a  self-administered  questionnaire  and  an  interview 
within  a  single  session.  During  this  session,  they  completed 
the  questionnaire  first,  in  about  30  minutes,  and  then  the 
interview,  in  about  90  minutes. 

Questionnaire  Method 

The  questionnaire  instrument  contained  three  sections  (see 
Appendix  A) : 

♦  Demographic  information.  The  first  section  of  the 
questionnaire  obtained  background  information  on  the 
contractor  personnel  regarding  their  prior  experience,  and 
their  roles  and  responsibilities  on  the  SST  programs.  The 
collected  data  were  presented  earlier  in  the  Background 
section. 

♦  Overview  items.  This  section  consisted  of  two  parts,  A 
and  B,  VTP/STRUCCTT  and  COBRAS /SGT,  respectively.  Each 
contained  identical  items  dealing  with  different  aspects 
of  the  programs.  The  contractor  personnel  were  instructed 
to  address  Parts  A  and/or  B  in  accordance  with  those 
programs  in  which  they  had  been  involved.  Those 
addressing  Part  B  who  had  been  involved  in  both  the  COBRAS 
and  SGT  programs  were  instructed  to  enter  comments 
differentiating  between  these  where  appropriate. 

♦  Constructivism  and  behaviorism.  This  section  was  also 
comprised  of  parts  A  and  B,  VTP/STRUCCTT  and  COBRAS/SGT, 
respectively.  Parts  A  and  B  each  contained  the  same  two 
groups  of  items.  The  first  group  described  elements  of 
constructivism,  while  the  second  group  described  elements 
of  behaviorism.  The  participants  were  instructed  to  check 
all  of  the  elements,  constructivistic  and/or 
behavioristic,  they  felt  represented  a  pervasive 
characteristic  of  the  programs  on  which  they  had  worked. 
The  constructivistic  and  behavioristic  elements  presented 
in  the  questionnaire  were  based  on  a  review  of  the 
literature.  Sources  for  these  elements  are  cited  in 
Shlechter  and  Finley  (in  preparation) . 

Interview  Method 


The  instruments  and  data  collection  procedures  for  the  CORs 
and  contractors  were  much  the  same.  The  few  differences  are 
described. 
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The  CORs 


Interviews  with  the  CORs  were  conducted  first  and  were 
exploratory  in  nature.  Each  COR  was  individually  interviewed 
several  times,  with  each  interview  lasting  between  one  and  two 
hours.  The  interviews  were  loosely  structured  vis-a-vis  the 
protocols  presented  in  Appendix  B.  Topics  discussed  during  the 
COR  interviews  included: 

♦  History  or  background  of  the  training  programs. 

♦  Goals  of  the  training  programs . 

♦  Instructional  design. 

♦  Implementation  concerns . 

♦  Lessons  learned. 

♦  Contributions  made  to  the  military  training  community. 

♦  Future  R&D  efforts. 

The  Contractors 


The  interviews  conducted  subsequently  with  the  contractors 
drew  upon  information  gained  from  the  CORs.  The  structured 
interview  form,  provided  in  Appendix  C,  contained  nine  items. 

The  first  item  was  a  follow-up  question  asked  regarding  any 
questionnaire  items  where  the  respondent's  answers  appeared 
unclear  or  incomplete.  Upon  completion  of  this  initial  phase  of 
each  interview,  the  interview  team  then  asked  participants  to 
discuss  and  offer  their  opinions  on  the  remaining  items 
contained  in  the  structured  interview  form.  The  remaining 
questions  probed  such  issues  as: 

♦  similarities  and  differences  between  the  VTP/STRUCCTT  and 
COBRAS /SGT  sets  of  programs  (asked  only  of  those 
respondents  who  had  worked  on  one  or  more  projects  in  both 
sets) ; 

♦  lessons  learned  from  these  programs; 

♦  contributions  made  by  these  programs  to  the  military 
training  community;  and 

♦  future  R&D  efforts  needed  to  improve  these  programs. 

In  each  case,  the  same  member  of  the  interview  team  asked  the 
questions  and  took  some  notes .  The  other  interviewer  always 
took  detailed  notes.  Both  interviewers  asked  participants  to 
elaborate  upon  any  of  their  comments  suggesting  additional 
insights  or  requiring  further  explanation.  The  only  interview 
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tape-recorded  was  the  first  one.  This  experience  provided 
evidence  that  tape-recording  was  not  necessary. 

Scoring  Procedures  and  Data  Analyses 

The  interview  data  were  scored  and  analyzed  using  the 
following  scheme: 

♦  Transcriptions .  The  person  taking  the  detailed  notes 
transcribed  the  participants'  comments.  The 
transcriptions  were  then  reviewed  and  any  discrepancies 
between  the  transcriptions  and  the  other  set  of  notes  were 
resolved. 

♦  Content  analyses .  The  transcribed  interview  data  were 
then  organized  into  content  categories  based  on  previous 
discussions  between  the  authors,  and  topic  areas  suggested 
by  the  interview  questions  and  responses.  Also  sorted 
into  these  categories  were  comments  written  by  the 
respondents  on  their  questionnaire  forms  and  their 
responses  to  the  initial  follow-up  queries .  The  data  were 
coded  in  order  to  protect  the  respondents '  privacy  and 
confidentiality.  The  content  categories  and  that 
organization  of  the  interview  data  were  not  directly  used 
in  presenting  the  findings  in  this  report.  However,  this 
exercise  greatly  facilitated  subsequent  discussions  and 
decisions  regarding  how  the  interview  responses  should  be 
organized. 

Data  Presentation 


As  a  part  of  above  process,  two  rules  were  developed 
concerning  how  the  interview  data  would  be  treated  and 
presented.  These  were: 

♦  Presentation  of  the  interview  data  would  be  descriptive  in 
nature,  but  not  to  the  extent  of  descriptive  statistics. 

♦  The  discussion,  in  all  cases,  would  focus  on  the  most 
salient  trends  and  comments. 

There  were  several  reasons  for  these  two  rules.  While  the 
interviews  were  structured,  the  questions  concerned  complex 
topics  where:  (1)  any  number  of  different  aspects  could  be  and 
were  addressed  on  each  topic;  and  (2)  the  respondents  differed 
from  each  other  with  regard  to  the  mix  of  programs  on  which  they 
had  worked  (see  Figure  2) ,  and  their  individual  mix  of 
activities,  vantage  points,  and  types  of  expertise.  Hence, 
there  was  generally  not  enough  commonality  between  the  aspects 
addressed  or  response  contents  to  allow  meaningful 
quantification  of  responses  across  the  key  developers.  The 
interviews  led  to  an  enormous  number  of  very  informative  and 
insightful  comments,  but  many  of  these  were  also  one  of  a  kind. 
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where  the  contractor  and  the  COR  key  developers  addressed 
the  same  topics  and  their  responses  indicated  similar 
perspectives,  their  comments  have  been  integrated.  Where, 
however,  the  CORs  or  contractors  addressed  different  topics  or 
offered  comments  reflecting  differing  perspectives,  these  data 
are  presented  separately.  The  COR  and  contractor  key  developers 
shared  considerable  experience  and  knowledge  but  there  were  some 
notable  differences  between  them.  The  contractor  personnel  had 
the  closer  day-by-day  technical  experiences.  The  CORs  had  the 
programmatic,  managerial.  Army,  and  R&D  perspectives 
corresponding  to  their  positions  and  responsibilities. 

RESULTS : 

USE  OF  THE  CONSTRUCTIVISM  AND  BEHAVIORISM  APPROACHES  FOR  SST 

It  was  stated  in  the  "Rationale"  section  of  the  Background 
that,  rather  than  adopting  an  oppositional  position  with  regard 
to  constructivism  and  behaviorism,  it  may  be  better  to  examine 
both  and  to  do  this  in  terms  of  specific  training  requirements. 
It  was  further  noted  that  the  overall  ARI  SST  R&D  program  could 
be  examined  as  two,  three,  or  four  programs  (VTP/STRUCCTT, 
SGT/COBRAS) .  As  there  are  some  major  differences  in  the 
training  requirements  addressed  across  these  SST  programs,  any 
differences  between  them  regarding  instructional  design 
approaches  would  be  of  interest.  If  these  differences  seemed  to 
also  reflect  differences  between  constructivism  and  behaviorism, 
this  would  be  especially  interesting. 

Similarities  and  differences  were  described  between  the  SST 
programs  during  the  interviews  with  the  contractor  key 
developers  (items  B.l  and  B.2,  Appendix  C) .  These  similarities 
and  differences  between  the  SST  programs  are  presented  first  to 
provide  background  information.  This  will  be  followed  by 
presentation  of  the  questionnaire  responses  describing  the  use 
of  constructivism  and  behaviorism  elements  across  the  SST 
programs  (Appendix  A,  Part  C) . 

SST  program  similarities  and  differences  were  addressed  by 
those  contractors  who  had  worked  on  more  than  one  program  (N  = 
11;  see  Figure  2) .  Data  describing  the  application  of 
constructivism  and  behaviorism  elements  to  the  SST  programs 
were  provided  by  all  15  contractors,  but  only  for  those  programs 
on  which  they  had  worked. 


Background 

Similarities  across  the  SST  Programs  (Interview  Item  B.l) 

The  majority  of  respondents,  whatever  the  combination  of 
programs  on  which  they  had  worked,  pointed  to  the  TSP  products 
as  being  the  desired  end  product  across  all  programs.  The 
respondents  identified  these  similarities  with  respect  to  TSPs: 

♦  training  strategy; 
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♦  development  procedures; 

♦  layout ;  and 

♦  content. 

Training  strategy.  Elements  of  strategy  included: 

♦  All  programs  focused  on  determining  the  training 
objectives  first,  and  then  structuring  and  designing  TSPs 
to  achieve  these  objectives. 

♦  Guidance  and  tools  were  provided  to  encourage  provision  of 
performance  feedback  (i.e.,  AARs)  pertaining  to  these 
objectives . 

♦  To  the  extent  possible,  a  "crawl -walk-run"  (c-w-r) 
approach  was  taken.  (This  training  model  is  used 
extensively  in  the  U.S.  Army.  In  the  SST  case,  it  was  the 
intentional  design  of  TSPs  to  match  the  skill  levels  of 
training  audiences  ranging  from  novice  to  more  expert.) 

♦  While  the, emphasis  differed  across  programs,  the  focus 
overall  was  more  on  process  than  on  mission  outcome. 

Development  procedures .  While  some  additional  procedures 
and  information  were  needed  for  COBRAS,  SAT  provided  the 
foundation  throughout.  The  design  of  the  scenarios  was 
structured  to  cause  particular  events  to  happen,  thereby  setting 
the  stage  for  achieving  the  intended  training  objectives. 

Layout.  Separate  sections  were  provided  in  the  TSPs  for 
the  exercise  controllers,  observer /controllers  (0/Cs) ,  and 
training  participants.  These  sections  were  tailored  to  the 
information  needs  of  each.  This  was  noted  by  one  respondent  as 
having  facilitated  exportation  of  the  SST  TSPs  to  other 
simulation  sites. 

Content.  The  same  missions  were  used  across  programs  with 
the  same  terrain  (central  corridor  of  the  National  Training 
Center  [NTC] ) ,  similar  operations  orders,  and  documentation 
describing  the  simulation. 

Differences  between  the  SST  Programs  (Interview  Item  B.2) 


Many  of  the  differences  noted  by  the  respondents  were  due 
to  improvements  over  time  (e.g.,  changes  in  TSP  format  based  on 
user  feedback)  and/or  changes  in  the  training  audiences  and, 
hence,  the  training  requirements  (see  Figures  la  and  lb) .  The 
respondents,  when  describing  differences  between  the  programs, 
used  VTP  as  their  referent  when  discussing  either  SGT  or  COBRAS 
This  being  the  case,  VTP  will  be  compared  to  SGT  first  and  then 
to  COBRAS. 
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Differences  between  VTP  and  SGT.  Most  of  the  differences 
noted  between  VTP  and  SGT  fell  into  three  general  categories: 
training  audience,  training  objectives,  and  Army  support. 
Comments  made  regarding  the  use  of  a  staff  processes  taxonomy 
will  also  be  described. 

The  SGT  training  audience  was  described  as  one  less 
experienced  than  that  of  the  VTP.  It  consisted  of  potential 
staff  officers  who  had  little  or  no  prior  experience  in 
performing  collective  staff  duties.  Further,  they  were  not  held 
to  established  training  evaluation  standards  as  few  standards 
exist  for  staff  processes.  In  contrast,  the  TSPs  for  the  VTP 
were,  in  large  part,  based  on  Army  Training  and  Evaluation  Plan 
Mission  Training  Plans  (ARTEP  MTPs)  providing  established 
standards  for  combat  performance. 

The  training  objectives  for  both  SGT  and  VTP  were  described 
as  being  appropriate  for  their  respective  training  audiences  and 
their  needs  (see  Figures  la  and  lb) .  It  was  noted  that  the  SGT 
TSPs  did  not  allow  flexibility  regarding  enemy  and  unit  actions 
as  did  the  VTP  TSPs .  The  SGT  was  given  high  marks  for  its  AARs . 
Respondents  pointed  to  both  the  quality  of  feedback  regarding 
training  objectives  and  the  SGT's  combined  use  of  small-group 
(i.e.,  section-level)  AARs  and  large-group  AARs. 

In  contrast,  the  VTP  AARs  were  described  as  sometimes 
evolving  into  a  tactical  discussion  rather  than  remaining 
focused  on  training  objectives.  This  occurred  despite  the  TSP 
guidance  and  tools  provided  for  the  VTP  AARs.  It  was  noted  that 
this  difference  in  AAR  tendencies  may  have  been,  in  part  at 
least,  a  function  of  the  VTP  being  more  tactically  focused. 
Further,  the  VTP  TSPs  were  based  more  on  such  traditional 
precedents  as  the  ARTEP  MTPs.  The  SGT,  in  contrast,  was  focused 
on  staff  processes  for  dealing  with  message  traffic  and  was  not 
tactically  focused  at  all.  In  addition,  as  the  SGT  was  an 
entirely  new  training  concept  for  the  combat  forces,  it  was  not 
encumbered  by  traditional  approaches. 

The  level  of  Army  support  for  the  VTP  as  compared  to  the 
SGT  was  described  summarily  by  one  respondent  as:  "The  level  of 
military  support  for  VTP  was  very  good;  it  was  very  bad  for 
SGT."  This  is  explained,  in  part,  by  these  comments:  the  SGT 
"...was  an  entirely  new  training  concept  for  the  combat  forces" 
and  "SGT  was  launched  without  a  safety  net  [i.e.,  there  were  no 
strong  sponsors]."  The  concept  for  SGT  was  based  on  earlier 
research  (Leibrecht,  Meade,  Schmidt,  Doherty,  &  Lickteig,  1994; 
Lickteig  &  Emery,  1994)  conducted  at  the  Armor  School.  However, 
few  representatives  of  the  Armor  School  involved  in  the  SGT 
projects  had  participated  in  or  had  knowledge  of  that  earlier 
research. 

Given  these  circumstances,  SGT  initially  encountered 
considerable  resistance.  It  is  worth  noting  that  acceptance  and 
support  did  grow  as  the  program  progressed  and  became  more 
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focused.  It  is  also  worth  noting  that  interest  has  since  been 
expressed  in  making  the  SGT  concept  a  part  of  the  Army's  Force 
XXI  Training  Program  for  the  21st  century. 

Differences  between  VTP  and  COBRAS.  Differences  noted  by 
the  respondents  when  comparing  VTP  to  the  COBRAS  programs  fell 
into  three  categories: 

♦  evolution  of  the  TSPs; 

♦  extent  to  which  these  R&D  efforts  were  exploratory  in 
defining  tasks  and  training  objectives;  and 

♦  mission  phases  and  simulation. 

The  differences  noted  between  the  TSPs  for  VTP  and  COBRAS 
were  improvements  as  would  be  expected  due  to  gaining  experience 
and  user  feedback  over  time.  For  example,  the  use  of 
"information  mapping"  (Horn,  1973)  as  a  part  of  the  TSP  format 
was  first  introduced  in  VTP's  SIMUTA-B  effort.  The  SIMUTA-B 
TSPs  were  also  described  by  the  contractors,  however,  as  "too 
huge."  One  contractor  key  developer  described  the  subsequent 
COBRAS  TSPs  as  gradually  becoming  "less  verbose."  The  same 
respondent  also  noted  that,  whatever  the  problems,  both  VTP  and 
COBRAS  TSPs  presented  the  key  information  and  that  this 
information  was  easy  to  find.  (For  further  details  regarding 
TSP  changes  over  time,  see  Shlechter  &  Finley  [in  preparation] ) . 

Some  respondents  pointed  to  the  more  exploratory  nature  of 
the  COBRAS  efforts  in  attempting  to  specify  training  objectives 
as  compared  to  those  of  the  VTP.  Specifying  objectives  is  a 
process  requiring  adequate  identification  and  descriptions  of 
the  tasks  to  be  performed  by  the  soldiers.  Action  tasks  and 
training  objectives  for  the  VTP  programs  were  identified  and 
fairly  well  described  a  priori  by  the  Army's  ARTEP  MTPs  and 
Situational  Training  Exercises  (STXs) ,  and  these  were  what  the 
Army  expected  to  be  used  in  development  of  TSPs.  The  primary 
VTP  R&D  questions  were  ones  of  how  to  best  use  this  information 
in  developing  SST  exercises,  and  then  how  to  construct  and 
format  TSPs  such  that  these  exercises  could  be  implemented  as 
intended. 

In  contrast,  many  recognized  that  battle  staff  tasks  were 
neither  completely  nor  well  defined.  Many  understood  that 
resolving  this  issue  might  be  a  necessary  first  step.  Indeed, 
the  COBRAS  and  SGT  programs  did  have  to  specify  staff  processes 
first,  define  these  as  tasks,  and  then  translate  these  tasks 
into  training  objectives  (cf.,  Shlechter  &  Finley,  [in 
preparation] ) . 

In  developing  their  TSPs,  COBRAS  team  members  were  able  to 
draw  upon  the  VTP  experience.  They  found  it  advantageous, 
however,  to  use  the  term  "performance  objectives"  and  create  a 
generic  list  of  these  objectives  for  the  COBRAS  TSPs.  The  need 
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for  this  generic  list  was  said  to  be  due  to  the  nature  of  staff 
processes  and  the  working  environment  of  cyclic  mission  phases. 
The  term  "training  objectives"  was  then  applied  to  those 
performance  objectives  selected  for  special  attention  during  an 
exercise . 

As  described  in  Figures  la  and  lb,  respondents  pointed  to 
VTP  as  having  focused  on  the  mission  phase  of  execution  while 
COBRAS  addressed  the  planning  and  preparation  phases  as  well. 
Addition  of  the  planning  and  preparation  phases  increased  the 
niimber  of  tasks  to  be  trained  and,  hence,  the  set  of  training 
requirements.  Additional  comments  were  that;  (1)  Only  the  last 
of  the  COBRAS  projects  was  truly  multi-echelon  with  respect  to 
training  participants  (i.e.,  training  participants  included 
personnel  at  both  battalion  and  brigade  echelons  within  a  single 
exercise) ;  this  is  in  contrast  to  using  role-players  or 
"training  aids"  for  echelons  above  or  below  the  echelon 
designated  for  training;  and  (2)  Only  COBRAS  developed  any 
exercises  that  did  not  require  use  of  a  software-driven 
simulation.  These  were  small  exercises  for  staffs  called 
"vignettes"  (cf.,  Shlechter  &  Finley,  [in  preparation]). 

Constructivism  and  Behaviorism/ SAT  Selections 
(Questionnaire  Part  III) 


Overall  differences  between  the  constructivism  and 
behaviorism/ SAT  approaches  were  discussed  earlier  and  listed  in 
Table  2.  In  contrast.  Table  3  uses  13  elements  to  present  a 
more  detailed  picture  of  differences  between  the  two  approaches. 
(The  contrasting  characterizations  are  based  on  an  extensive 
review  of  the  literature.  See  Shlechter  &  Finley  [in 
preparation]  for  citations.) 

The  characterizations  in  Table  3  are  the  ones  presented 
in  Part  III  of  the  Contractor  Questionnaire  (see  Appendix  A)  . 

The  respondents  were  asked  to  select  those  elements  which  had 
been  applied  to  each  of  the  program  sets  on  which  they  had 
worked.  Although  they  were  informed  that  there  was  some 
controversy  about  the  concepts,  they  were  instructed  to  select 
all  elements  that  had  been  applied,  but  only  those  elements. 

They  were  told  that,  if  appropriate,  they  could  select  both  the 
constructivistic  and  behavioristic  versions  of  an  element. 
Section  A  of  the  questionnaire  requested  these  judgments  for  the 
VTP/STRUCCTT  programs,  while  Section  B  requested  the  same  for 
the  SGT  and/or  COBRAS  programs. 
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Table  3 .  Elements  of  the  constructivism  and  behaviorism 
instructional  theories. 


INSTRUCTIONAL  THEORY  ELEMENTS 

CONSTRUCTIVISM 

BEHAVIORISM 

1 

Training  objectives  determined 
by  the  training  participants. 

Training  objectives  determined 
by  the  instructional  design 
team. 

2 

Training  objectives  also  emerge 
as  training  participants 
interact  with  the  training 
materials . 

Training  objectives  determined 
as  an  initial  part  of  the 
design  process. 

3 

Task(s)  immerse (s)  participants 
in  realistic  battlefield 
conditions  for  their  echelon. 

Task(s)  does  not /do  not  need  to 
immerse  training  participants 
in  realistic  battlefield 
conditions  for  their  echelon. 

4 

Course  materials  focused  on 
developing  a  unit's  higher 
order  cognitive  skills  (e.g., 
its  tactical  decision-making) . 

Course  materials  focused  on 
developing  the  unit ' s 
procedural-level  tactical 
skills  (e.g.,  executing 
tactical  formations) . 

5 

Course  materials  focused  on 
helping  participants'  develop 
the  skills  necessary  to  fight 
in  new  and  different 
battlefield  conditions. 

Course  materials  focused  on 
helping  participants'  develop 
the  skills  necessary  to  fight 
in  battlefield  conditions  which 
resemble  the  scenario. 

6 

Instructional  program  need  not 
contain  a  standardized  set  of 
instructional  materials. 

Instructional  program  must 
contain  a  standardized  set  of 
instructional  materials. 

7 

Instructional  program  does  not 
contain  a  particular 
instructional  sequence  (e.g., 

" crawl -walk-run " ) . 

Instructional  program  contains 
a  particular  instructional 
sequence  (e.g.,  "crawl -walk- 
run  " )  . 

(table  continues) 
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Table  3  (Continued) 


CONSTRUCTIVISM 

BEHAVIORISM 

8 

Instructional  materials 
developed  for  the  more 
experienced  or  advanced 
training  participants. 

Instructional  materials 
developed  for  the  less 
experienced  or  novice- level 
participants . 

9 

Experiential  learning  is  more 
important  than  mastery 
learning. 

Mastery  learning  is  more 
important  than  experiential 
learning . 

10 

Instructional  personnel  should 
refrain  from  providing 
performance  feedback  to  the 
participants  as  they  are 
executing  a  table. 

Instructional  personnel  should, 
if  needed,  provide  performance 
feedback  to  the  participants  as 
they  are  executing  a  table. 

11 

Student-led  AARs . 

Instructor-led  AARs. 

12 

Feedback  geared  more  to  the 
unit  processes  (e.g., 
communication  among  tanks) 
associated  with  any  particular 
action  (e.g.,  getting  to  the 

Line  of  Departure (LD) /Start 

Point  (SP)  on  time)  than  to  the 
action  itself. 

Feedback  geared  more  to  the 
unit's  actions  (e.g.,  getting 
to  the  LD/SP  on  time)  than  to 
the  processes  (e.g., 
communication  among  tanks) 
associated  with  its  action(s). 

13 

A  non-linear  or  spiral 
progression  used  in  the 
instructional  design  process. 

A  linear  or  spiral  progression 
used  in  the  instructional 
design  process. 

Three  issues  were  of  interest: 

♦  Which  of  these  elements  were  used  in  developing  and 
implementing  the  SST  programs? 

♦  Were  there  overall  differences  between  the  VTP/STRUCCTT, 
SGT,  and  COBRAS  programs  regarding  their  use  of  the  two 
instructional  approaches? 

♦  Were  there  differences  between  the  VTP/STRUCCTT,  SGT,  and 
COBRAS  programs  regarding  which  elements  were  used? 
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Presentation  of  the  Data 


To  address  these  issues,  the  respondents'  selections  of 
elements  were  grouped  into  a  matrix  of  cells  defined  by  these 
three  factors:  the  13  elements;  two  instructional  approaches; 
and  three  SST  program  sets.  The  number  of  participant  responses 
possible  in  a  matrix  cell  for  either  VTP/STRUCCTT,  SGT,  or 
COBRAS  were,  respectively,  N  =  12,  5,  and  8. 

(Table  1  and  Figure  2  indicated  that  the  N  for  SGT  was  7, 
where  two  of  these  persons  also  worked  on  the  COBRAS  program. 

In  completing  this  particular  questionnaire  section,  these  two 
persons  indicated  that  the  elements  they  selected  were 
appropriate  for  both  COBRAS  and  SGT.  A  decision  was  made  to 
count  their  responses  only  once,  assigning  them  to  the  COBRAS 
column.  Hence,  the  SGT  N  =  5  in  this  instance,  rather  than  the 
N  =  7  used  elsewhere.) 

This  matrix  is  presented  in  Table  4  with  the  response  data 
displayed  in  two  formats:  (1)  number  of  persons  choosing  this 
element  (unboldened)  and  the  proportion  choosing  it  (boldened) . 

The  proportions  shown  are  the  frequency  with  which  it  was  selected 
divided  by  the  maximum  number  possible  for  that  program  (maximum  Ns 
=12,  5,  and  8) .  The  summary  proportions  shown  for  each  program  for 
each  of  the  two  instructional  approaches  are  the  responses  summed 
across  the  13  elements  and  then  divided  by  the  maximum  sum  possible 
for  that  column.  The  maximum  possible  sxom  for  VTP/STRUCCTT  was  13  x 
12  =  156;  for  SGT,  13  x  5  =  65;  and  for  COBRAS,  13  x  8  =  104. 

The  Findings 


Similarities  across  the  SST  programs.  Table  4  presents  the  data 
for  all  15  respondents  concerning  the  constructivist  and  behaviorist 
aspects  of  SST  R&D  programs  on  which  they  had  worked.  This  table 
provides  insights  into  the  instructional  design  characteristics  of 
ARI's  SST  programs.  With  regard  to  similarities,  there  were  six 
SAT/behaviorism  elements  that  were  selected  across  all  SST  programs 
by  60%  or  more  of  the  respondents.  These  six  elements  are 
highlighted  in  gray  in  Table  4  and  include: 

♦  The  SST  instructional  design  team  determines  training 
objectives  and  does  so  early  in  the  design  process  (elements  1 
and  2) . 

♦  The  SST  program  must  contain  standardized  materials  (element 
6)  . 

♦  The  SST  program  must  contain  a  particular  sequence  (element 
7)  . 

♦  Instructional  materials  are  developed  for  more  novice 
participants  (element  8) . 
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Table  4.  Frequency  and  proportions  of  selections  per  constructivism  and  SAT /behavior ism 
element  across  the  VTP/STRUCCTT  (VTP/STR) ,  SGT,  and  COBRAS  sets  of  programs 

Legend :CZI]  Highlights  proportions  of  .60  or  higher  across  the  three  sets  of  training  programs. 

Mi  Highlights  program  proportions  which  are  substantially  different  than  those  of  the  other  program(s). 
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♦  Instructional  personnel  should,  if  needed,  give  feedback  during 
an  exercise  (element  10) . 

Hence,  a  similarity  across  the  SST  programs  was  the  greater  use  of 
the  SAT /behaviorism  approach  to  instructional  design.  The  use  of 
this  approach  is  further  indicated  by  Table  4's  summary  data  for 
behaviorism/ SAT  (.69,  .63,  and  .58). 

Differences  between  the  SST  programs.  Table  4's  data  also 
indicate  several  key  differences  between  VTP/STRUCCTT,  SGT,  and 
COBRAS  with  regard  to  which  of  the  behaviorism  and  constructivism 
elements  were  applied.  These  differences  are  highlighted  in  Table 
4  by  using  contrasting  black  and  white  cells  (e.g.,  for  element  4, 
note  the  values  of  .80  and  1.00  for  SGT  and  COBRAS,  and  .83  for 
VTP/STRUCCTT).  These  key  differences  are  as  follows: 

♦  The  VTP  materials  tended  to  focus  upon  training  units  to 
develop  their  procedural  skills,  while  the  COBRAS  and  SGT 
materials  tended  to  focus  upon  training  units  to  develop 
their  cognitive  skills  (element  4) . 

♦  Experiential  learning  was  more  important  for  the  COBRAS 
programs  than  for  the  VTP/STRUCCTT  programs;  mastery 
learning  was  much  less  a  focus  of  the  COBRAS  program  than 
for  the  VTP/STRUCCTT  programs  (element  9) . 

♦  Feedback  focused  more  on  a  unit's  actions  in  the 
VTP/STRUCCTT  programs,  whereas  feedback  for  the  COBRAS 
programs  focused  more  on  tactical  processes  (element  12). 

♦  A  non-linear  instructional  design  process  was  more 
pronounced  for  the  COBRAS  R&D  efforts.  In  contrast,  the 
use  of  non-linear  and  linear  processes  in  VTP/STRUCCTT 
efforts  appeared,  to  the  respondents,  to  be  balanced 
(element  13). 

While  SGT  tended  more  toward  behaviorism/ SAT  selections,  it 
did  not  consistently  line  up  with  the  differences  noted  between 
VTP/STRUCCTT  and  COBRAS.  Given  the  uniqueness  of  SGT  as  a 
program  designed  to  transition  soldiers  from  individual  to 
collective  training  for  staff  positions,  its  alignment  with  the 
two  instructional  approaches  appears  reasonable. 

Constructivism  and  SST.  The  use  of  constructivism  in 
designing  and  developing  the  SST  TSPs  is  also  evident  in  Table  4, 
although  to  a  lesser  degree.  Most  of  the  constructivism  elements 
received  a  reasonable  number  of  responses  across  the  SST  programs . 
Moreover,  the  constructivism  and  behaviorism  elements  each  received 
nearly  the  same  number  of  responses  for  the  COBRAS  training 
programs.  Hence,  the  different  SST  efforts  reflected  both  the 
behaviorism  and  constructivism  approaches  to  instructional  design, 
with  the  latter  approach  being  more  evident  for  the  COBRAS  R&D 
efforts . 
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Another  note  concerning  the  constructivism  approach  can  be 
made  based  on  examination  of  the  proportions  of  respondents 
choosing  elements  1  and  2  under  the  constructivism  column  in 
Table  4.  While  the  predominant  use  of  SAT/behaviorism' s 
elements  1  and  2  was  noted  earlier,  several  respondents  also 
chose  the  constructivistic  version  of  elements  1  and  2 .  When 
queried  about  their  choices,  the  respondents  said  that  some 
units  were  observed  to  play  at  least  some  role  in  determining 
their  training  objectives.  They  described  that  role  as  being 
one  of  reviewing  the  available  TSPs  before  initiating  their 
training,  selecting  the  ones  they  felt  to  be  most  appropriate. 
And,  at  times,  training  participants  did  identify  additional 
objectives  after  completing  an  SST  exercise.  These  additional 
objectives  were  described  as  emerging  when  units  recognized 
indications  of  unanticipated  skill  deficiencies  and/or  gained  a 
better  understanding  of  the  simulation's  capabilities. 

A  Tangential  but  Interesting  Finding 

Given  the  non-homogenous  composition  of  the  respondent 
population,  the  data  were  also  examined  during  analysis  to 
determine  if  there  were  any  patterns  reflecting  individual 
respondent  differences  beyond  the  anticipated  differences 
between  elements  and  between  SST  programs.  An  interesting 
difference  was  found  within  the  group  of  respondents  who  had 
worked  on  more  than  one  program,  of  which  there  were  11  (see 
Figure  2)  .  Approximately  half  of  them  (N  =  6)  provided 
different  instructional  theory  selections  for  each  of  the  two  or 
three  programs  on  which  they  had  worked  for  some  of  the  13 
elements.  Further,  these  choices  appeared  to  be  reasonably 
consistent  across  the  6  respondents.  The  other  half  (N  =  5) 
made  different  selections  for  each  program  on  no  more  than  one 
element,  if  that.  The  data  on  these  two  groups  were  compared 
and  it  was  determined  that  some  background  differences  existed 
which  might  explain  this  tendency,  on  the  part  of  some 
respondents,  to  differentiate  between  programs.  Given  this 
finding,  the  background  data  and  instructional  theory  choices  of 
those  persons  we  will  call  the  "differentiators"  -  as  compared 
to  the  "non-differentiators"  -  are  also  presented,  but 
separately  in  Appendix  D.  This  is  intended  to  allow  readers,  if 
they  are  so  inclined,  to  decide  for  themselves  which  data  set 
best  answers  the  issues. 

The  data  trends  for  the  differentiators  were  found  to  be 
the  same  as  they  were  for  the  entire  group  of  15,  only  stronger 
(i.e.,  the  patterns  of  proportions  appearing  in  Table  4  and 
Table  D-2  are  quite  similar) .  The  data  from  the 
differentiators,  despite  the  very  small  Ns,  are  of  interest 
because:  (1)  They  confirm  the  trends  appearing  in  Table  4  and 

(2)  They  reaffirm  the  value  of  examining  the  effects  of 
individual  differences  whenever  feasible. 
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RESULTS : 

NEEDS  FOR  AN  SST  DEVELOPMENT  MODEL,  FRONT  END  ANALYSES, 
AND  RESEARCH  TO  IMPROVE  THE  APPLICATION  OF  SST 

An  SST  Development  Model 


The  more  tightly  a  scenario  is  detailed  and  controlled  with 
regard  to  closely  defined  training  objectives,  the  more 
"structured"  a  training  exercise  becomes.  The  COR  key 
developers  presented  the  issue  of  how  much  structure  is  needed 
as  one  moves  to  different  echelon  and/or  skill  levels.  The 
questions  were: 

♦  To  what  extent  can  or  should  structure  be  imposed? 

♦  How  is  it  best  accomplished? 

The  term  "model"  is  used  here  to  suggest  a  conceptual 
framework  and  knowledge  which  could  guide  SST  TSP  development 
activities  related  to  the  above  questions.  That  guidance  could 
concern  such  issues  as  choice  of  instructional  design  features, 
scenario  scripting,  and  matching  skill  levels  to  simulation 
capabilities . 

Instructional  Design  Elements 


Contrasts  between  behaviorism/SAT  and  constructivism  can 
be  an  important  consideration  in  designing  TSPs.  Some  have 
argued  that  the  behaviorism/SAT  approach  is  more  useful  for 
procedural  and  mechanical  skills,  while  constructivism  is  more 
appropriate  for  leadership  and  cognitive  skills.  One  finding 
from  review  of  the  SST  programs  (Shlechter  &  Finley,  [in 
preparation]  )  and  confirmed  by  data  collected  for  this  report 
(see  Table  4)  has  been  that  application  of  design  elements 
selectively  chosen  from  both  approaches,  and  perhaps  others  as 
well,  may  be  a  more  effective  approach.  A  clearer  understanding 
is  needed  regarding  which  instructional  design  elements  would  be 
most  appropriate  under  what  conditions. 

Scenario  Scripting 


Key  developers  described  the  higher  echelons  as  more 
complex  and  difficult  to  control.  They  noted  that  this  is 
exacerbated  by  the  fact  that  the  software  currently  used  to  run 
training  simulations,  especially  ones  for  higher  echelons,  can 
neither  be  scripted  easily  in  accordance  with  the  details  of  an 
SST  scenario  nor  easily  modified.  The  CORs  suggested  that  a 
review  of  instructional  design  elements  like  those  in  Table  4 
and  a  sample  of  SST  TSPs  might  be  made  in  conjunction  with  an 
examination  of  current  simulation  programming  languages  and 
techniques .  The  goal  would  be  to  identify  enhanced  scripting 
techniques  and/or  technologies  which  could  afford  greater 
flexibility  in  controlling  scenarios. 
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Matching  Skill  Levels  to  Simulation  Capabilities 


Large  and  complex  constructive  simulations  like  BBS  are 
expensive  to  run  from  the  standpoints  of  needed  support 
personnel  and  facilities.  A  concern  expressed  by  the  CORs  was 
that  such  simulations  are  not  only  costly,  they  are  not  very 
effective  if  the  soldiers  have  not  yet  acquired  the  necessary 
"gate"  skills.  Gate  skills  are  those  that  need  to  be  acquired 
first  before  attempting  to  perform  the  collective  tasks 
appropriate  for  training  in  a  BBS  environment.  The  question  the 
CORs  raised  was,  "To  what  extent  might  more  extensive  use  of 
inexpensive  small  group  staff  exercises,  similar  in  concept  to 
crew  drills  for  a  platoon,  prepare  soldiers  to  then  receive 
maximum  training  benefits  from  a  BBS  exercise?"  The  CORs 
suggested  that  such  exercises  should  also  lend  themselves 
readily  to  effective  structuring. 

Front  End  Analyses  (FEAs) 


One  COR,  when  asked  to  identify  "the  top  ten  tricky  SST 
issues,  "  did  name  ten  of  them.  For  five  of  these,  most  of  them 
at  the  top  of  the  list,  the  COR  recommended  conducting  FEAs  to 
resolve  them.  If  FEAs,  also  commonly  referred  to  as  "needs 
analyses,  "  were  to  be  conducted  then  the  results  from  these 
should  serve  two  purposes: 

♦  provide  a  foundation  for  an  Army  SST  Master  Plan;  and 

♦  identify  those  tasks  constituting  the  minimum  essential 
set  of  tasks  needing  to  be  trained,  the  nature  of  their 
training  requirements,  and  the  best  training  strategies 
for  them. 

Such  FEAs  were  described  by  the  CORs  as  being  important  avenues 
toward  fully  realizing  SST  benefits  and  doing  this  in  the  most 
efficient  manner. 

Establish  a  Foundation  for  an  Army  SST  Master  Plan 

Both  CORs  made  many  comments  regarding  how  they  wished  they 
had  been  able  to  start  at  the  beginning  of  their  SST  R&D,  with 
the  knowledge  they  have  now,  to  develop  plans  of  greater  scope 
and  to  do  so  within  a  larger  Army  context .  They  wished  that 
they  had  been  able,  much  earlier,  to  understand  the  need  to  more 
completely  consider  the  "big  picture"  and  conceptualize  at  a 
macro-level.  They  suggested  several  ideas  which,  if  tied 
together,  might  well  constitute  an  outline  for  a  "Master  Plan" 
for  SST.  The  planning-related  needs  for  FEAs  were  described  as 
ones  providing  a  better  definition  and  organization  of  the 
Army's  overall  training  requirements,  and  approaches  to 
satisfying  these  requirements. 

A  caution  was  offered  by  the  CORs  regarding  any  FEAs  that 
might  be  done  to  provide  a  framework  supporting  integrated 
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development  and  use  of  TSPs  across  echelons .  These  FEAs  should 
start  at  the  corps  echelon  and  work  down  to  platoon  level. 
Because  this  was  not  done  in  the  VTP  program  (i.e.,  an  FEA  was 
not  available  and  TSPs  were  first  developed  for  the  platoon 
level) ,  the  current  set  of  VTP  TSPs  cannot  be  easily  used  in 
combination  with  COBRAS  TSPs,  which  were  developed  for  brigades 
responding  to  division  orders. 

Identify  Minimum  Essential  Set  of  Tasks  for  Training 

FEAs  were  also  suggested  to  identify:  those  tasks 
constituting  the  minimum  essential  set  of  tasks  needing  to  be 
trained;  the  nature  of  the  training  requirements  for  these 
tasks;  and  the  best  training  strategies  for  them.  We  suggest 
that  an  intent  to  perform  these  FEAs  could  be  placed  in  the 
proposed  Army  SST  Master  Plan. 

The  CORs  recommended  that  these  more  specifically  focused 
FEAs  be  performed  within  the  context  provided  by  the  overall 
picture (s)  of  training  requirements  discussed  above.  These  FEAs 
could  focus  on,  for  example,  a  particular  type  of  unit  (e.g.,  an 
armor  unit)  and  attempt  to  determine  for  that  unit:  what  are 
the  most  essential  tasks  to  be  trained  and  what  should  the 
progression  of  training  be  (e.g.,  sequence,  repetitions,  c-w-r 
events,  mission  and  environmental  contexts).  These  requirements 
could  then  be  matched  to  what  can  be  done  in  the  available 
virtual,  constructive,  and  live  simulations. 

As  FEAs  identifying  the  minimum  set  of  essential  tasks  for 
training  are  completed  for  a  unit,  these  analyses  could  be 
extended  to  functions  requiring  external  coordination  (e.g., 
requesting  indirect  fire  support  from  another  unit  or  higher 
headquarters) .  A  desirable  goal  would  be  to  identify  5-10 
sets  of  the  most  essential  skills  on  which  training  would  focus 
in  a  unit.  For  example,  integration  of  direct  and  indirect  fire 
skills  might  be  identified  as  one  of  the  essential  sets. 

Both  CORs  mentioned  Lieutenant  General (R)  Brown's  concept 
of  a  three  step  training  strategy  which  has  been  published  as 
part  of  a  U.S.  Army  Training  and  Doctrine  Command  (TRADOC) 
memorandum  (U.S.  Army  Training  and  Doctrine  Command,  1998) ,  The 
three  steps  are:  (1)  learn  the  basics  (e.g.,  warfighting  skills 
like  gunnery);  (2)  become  proficient  in  the  tasks,  conditions, 
and  standards  of  needed  hardware  and  software;  and  (3)  develop 
highly  adaptive  hyperproficient  individuals,  small  teams,  and 
units.  Some  of  the  SST  key  developers  along  with  some  others, 
in  discussing  the  three  step  strategy,  expressed  opinions  that 
even  completing  step  2  training  to  the  point  where  they  are  able 
to  enter  step  3  training  would  be  very  difficult  in  most  cases. 
This  is  due  to  resource  limitations  and  other  constraints  facing 
most  units.  One  COR  suggested,  however,  that  if  FEA  were  able 
to  establish  small  enough  sets  of  minimum  essential  skills  and 
if  research  also  established  ways  to  primarily  train  those 
skills  alone  (e.g.,  through  repeated  use  of  varied  vignettes  and 
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TSPs)  such  that  soldiers  could  adequately  transfer  and  adapt  to 
situations  requiring  other  tasks  to  be  performed  -  then  a  step  3 
level  of  expertise  might  be  attainable  on  those  sets  at  least. 
This  COR  suggested  that  this  would  take  the  concept  of 
structured  training  a  step  further  -  train  and  test  on  only  a 
small  set  until  the  soldiers  are  indeed  experts  on  these. 

Transfer  of  Training  Methods  Research 

Whatever  the  training  or  education  being  provided,  three 
key  questions  are:  (1)  Will  it  contribute  to  soldiers'  ability 
to  perform  effectively  in  future  instructional  and/or 
operational  endeavors?;  (2)  To  what  extent  will  this 
contribution  cover  a  variety  of  events  and  surrounding 
conditions?;  and  (3)  Is  it  an  efficient  means  for  doing  this?_ 

If  the  answer  to  the  first  question  is,  "yes,*  then  the  training 
is  said  to  have  "positive  transfer."  If  the  answer  to  the 
second  question  is,  "positive  transfer,  but  only  to  one  type  of 
situation,"  then  the  transfer  is  limited  as  opposed  to  general, 
or  generic.  If  the  answer  is  not  only,  "no,"  but  that  it  hinders 
performance  as  well,  then  the  training  is  said  to  have  "negative 
transfer . " 

Several  aspects  of  training  transfer  related  to  SST  will  be 
discussed  here:  the  "nesting"  concept,  adaptive  thinking, 
training  bridges,  and  training  measurement.  Except  training 
measurement,  the  translation  of  all  of  these  from  research 
findings  into  applications  would  be  in  the  form  of  training 
methods  or  techniques.  Good  measures  of  training  results  are 
necessary  tools  for  both  research  and  application,  providing  the 
needed  diagnostic  and  evaluative  information.  These  topics  are 
closely  interrelated  and  they  could  affect  the  extent  to  which 
SST  transfers  positively,  especially  to  operational 
environments . 

The  Nesting  Concept 

Some  key  developers  suggested  that  research  was  needed  into 
the  relative  benefits  of  "nesting"  missions  and  orders  versus 
using  different  missions  and  scenarios.  The  term  "nesting"  is 
defined  as  using  the  same  missions  and  operations  orders  at  each 
echelon.  (Terrain  was  not  an  explicit  part  of  the  nesting 
definition  because  the  central  corridor  of  NTC  was  the  only 
terrain  available  during  ARI's  SST  R&D.)  The  intent  of  nesting 
is  to  enable  a  tank  platoon,  for  example,  to  train  separately 
using  a  TSP  for  a  defense  mission,  and  then  to  train  again, 
later,  as  a  platoon  within  a  company  performing  the  same 
mission.  In  each  case,  a  defense  mission  would  be  executed 
using  the  same  orders,  tailored  only  as  needed  to  reflect 
differences  between  platoon  and  company  echelon 
responsibilities.  One  contractor  described  the  good  and  bad 
features  of  nesting  as  being:  "The  good  feature  of  nesting  is 
it  gives  a  common  framework  across  echelons  and  probably 
enhances  training  effectiveness  as  well  as  efficiency.  The  bad 
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feature  is  that  people  think  that  by  nesting,  what  is  done  in 
one  environment  will  necessarily  transfer  to  any  other 
environment.  This  may  not  be  true  because  changes  in  mission, 
enemy,  terrain,  troops,  and  time  available  (METT-T)  does  change 
performance  requirements . *  The  concern  of  several  respondents 
was  that  more  TSP  variations  in  mission,  scenario,  and  operating 
environment  may  be  needed  to  ensure  transfer  of  training.  Their 
question,  as  stated  by  one,  was;  "How  much  variety  is  enough?" 

Adaptive  Thinking 

A  concept,  addressed  earlier  in  discussing  FEAs,  was 
whether  there  might  be  some  minimum  number  of  essential  task 
skill  sets  on  which  training,  with  varied  vignettes  and  TSPs, 
could  focus  on  a  continuing  basis.  It  was  suggested  by  one  COR 
that  -  if  it  were  determined  that  continued  training  on 
primarily  these  tasks  would  "provide  adequate  transfer  and 
adaptation  to  situations  requiring  other  tasks  to  be  performed  - 
then  Step  3  expert  performance  levels  could  be  achieved. " 

Adaptive  thinking  is  a  skill  beyond  simple  recognition  and 
reaction  to  similarities  between  training  and  operational 
environments.  It  consists  of  having  learned  how  to  adapt  one's 
knowledge  to  new  situations  and  how  to  identify  what  additional 
information  is  needed  (Finley,  Sanders,  &  Ryan,  1996) .  Adaptive 
thinking  is  thus  a  proactive  component  of  training  transfer.  If 
minimiom  sets  of  necessary  skills  were  identified,  then  research 
to  establish  a  means  of  training  this  proactive  aspect  of 
transfer  of  training  might  increase  the  benefits  of  SST  even 
further . 

Training  Bridges 


Another  aspect  of  training  transfer  addressed  by  one  key 
developer  was  how  to  enable  soldiers  to  understand  and 
appreciate  how  their  previous  virtual  and  constructive 
simulation-based  training  applies  to  their  live  training, 
especially  Combat  Training  Center  (CTC)  rotations  like  NTC  and 
Joint  Readiness  Training  Center  (JRTC) .  Soldier  training 
participants  have  been  observed  to  question  the  value  of  their 
prior  virtual  and  constructive  simulation-based  training  when 
they  experience  things  at  CTCs  not  experienced  before  or  in  the 
same  manner. 

The  problem  here  may  be  one  of  CTCs  being  viewed  as  "the 
real  world;"  that  is,  as  being  much  closer  to  combat  reality 
than  is  actually  the  case.  In  fact,  the  CTCs  are  simulations  as 
well.  They  too  are  limited  in  the  realities  they  can  simulate 
and  these  limitations  are  not  always  apparent  to  many  training 
participants.  The  bottom  line  is  that  R&D  is  needed  to 
determine  means  for  enabling  soldiers  to  better  link  and 
appreciate  their  CTC  and  other  simulation-based  training 
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experiences;  and  to  understand  the  ways  in  which  they  each 
differ  from  true  reality. 

Training  Measurement 

Good  measures  of  performance  are  key  to  effective  AARs  in 
both  evaluating  the  performance  of  training  participants  and 
diagnosing  their  problem  areas.  However,  as  one  COR  observed, 
“We  never  succeeded  in  finding  an  evaluation  form  that  the  0/Cs 
liked  and  would  use  without  prodding."  The  COR  suspected  it  was 
not  only  the  filling  out  of  forms  that  the  0/Cs  disliked,  but 
that  it  may  also  have  reflected  their  opinions  regarding  the 
utility  of  the  information  gained.  A  research  need  identified 
by  both  CORs  concerned  the  requirements  for  measures  that  would 
provide  data  more  useful  to  the  AAR  process.  They  also 
emphasized  the  need  to  automate,  to  the  fullest  extent  possible, 
the  collection  and  analysis  of  data  obtained  using  these 
measures . 

Better  (i.e.,  more  useful)  measures  of  performance  are 
needed  not  only  for  training  AARs,  they  are  also  needed  in 
efforts  to  assess  operational  performance.  To  the  extent  good 
measures  of  performance  can  be  identified  for  AAR  purposes,  they 
may  also  be  found  useful  in  efforts  to  evaluate  the  extent  and 
nature  of  training  transfer  to  the  operational  environment. 

RESULTS : 

ARMY  SST  MASTER  PLAN,  LOGISTICS  SYSTEM,  AND  POLICY 

In  the  Rationales  and  Issues  section,  it  was  indicated  that 
one  of  the  purposes  of  this  report  was  to  identify  insights  into 
how  to  make  SST  a  regular  part  of  Army  training.  Based  on 
analyses  of  the  information  obtained  from  ARI ' s  key  SST 
developers,  recommendations  for  an  Army  SST  Master  Plan  and 
Logistics  System,  along  with  related  policies,  will  be  described 
in  this  section.  Prior  to  presenting  these  recommendations, 
however,  information  drawn  from  the  key  developers  will  be 
provided  as  background. 


Background 

SST  Contributions  to  the  Military  Training  Community 

SST  training  benefits  were  described  by  contractor  key 
developers  as  being  the  major  contribution  and  one  that  was 
"validated."  The  term,  "validated,"  was  used  to  convey  personal 
opinions  based  on  direct  and  repeated  observations  of  unit 
performance  improvements  resulting  from  the  use  of  SST  TSPs . 
These  improvements  were  observed  during  formative  evaluations  of 
TSP  prototypes  and,  in  some  cases,  subsequent  operational  uses 
of  the  TSPs.  Several  contractors  went  on  to  describe  the 
training  benefits  from  a  larger  perspective  of  training 
simulations  in  general.  This  general  benefit  was  summarized  by 
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one  as:  "The  Army  was  shown  how  to  use  their  simulations  so  as 
to  realize  their  training  benefits  rather  than  wasting  them." 

Observations  of  training  effectiveness  were  noted  for  all 
the  SST  programs:  VTP,  COBRAS,  and  SGT  TSPs .  Improvements  in 
performance  were  most  explicitly  described  for  the  platoon  and 
company  VTP  TSPs,  where  actions  and  processes  are  more  discrete 
and  directly  observable.  One  succinct  siimmary  was  that  units 
progressed  from  being  "...unproficient  and  disorganized  to 
becoming  much  better."  Another  respondent  estimated  that,  "The 
'light  bulb  goes  on'  after  completion  of  TSPs  for  about  80%  of 
the  units." 

Evidence  that  there  is  a  growing  appreciation  of  SST 
benefits  was  a  recent  policy  statement  on  the  part  of  the  U.S. 
Army  Training  and  Doctrine  Command  (TRADOC) .  This  statement 
described  the  strategy  to  be  used  in  developing  exercises  for 
digital  instruction  (TRADOC,  1998).  It  cited  SST's  SGT  and 
COBRAS  programs  as  the  development  models  to  be  followed. 

SST  Problems 


Nature  and  value  of  SST  are  not  widely  known.  Despite  the 
TRADOC  memorandum  noted  above,  knowledge  and  appreciation  of  SST 
are  not  widely  held  across  the  Army.  Army  personnel  who  have 
had  very  limited  or  no  experience  with  SST  TSPs  find  it 
difficult  to  fully  understand  or  appreciate  the  value  of  SST. 

One  key  developer  succinctly  summarized  the  requirement  as: 
"Everyone  must  understand  that  structured  training  is  different 
-  it  is  not  'free  play'  nor  is  it  'bonding  with  the 
generators.'"  The  problem  was  described  as  one  of  "...trying  to 
get  folks  to  truly  understand  structure  and  its  real  value,  and 
to  share  in  the  larger  training  picture." 

One  COR  summarized  the  situation  by  asking  the  question: 
"For  SST,  the  Army  has  actually  had  to  relearn  SAT  -  how  do  we 
avoid  this  with  SST  as  well?"  Both  CORs  and  contractors 
expressed  concern  that  the  understanding  of  training  that  many 
in  the  Army  gained  though  the  SST  R&D  programs  could  easily  be 
lost.  The  SAT  concept  and  AAR  procedures  were  pointed  to  as 
examples  of  where,  even  though  these  approaches  were  first 
developed  and  formally  adopted  by  the  Army,  this  original 
knowledge  base  has  not  been  maintained.  Many  now  in  the  Army, 
while  using  the  SAT  and  AAR  acronyms  and  believing  they  are 
following  the  regulatory  guides,  are  actually  only  following 
examples  they  have  observed.  These  examples,  unfortunately, 
have  often  been  poor  ones.  While  some  soldiers,  seemingly 
through  instinct,  are  excellent  training  developers  and 
executors,  most  others  need  to  be  tutored  and  then  receive 
guided  experience.  Providing  this  type  of  instruction  is  not  a 
current  practice  for  SAT  or  AARs,  let  alone  SST. 

Enjoying  "free-play".  The  VTP  was  the  first  of  the  SST  R&D 
programs  and  was  directed  to  apply  the  capabilities  of  SIMNET  to 
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a  serious  training  problem  (see  Shlechter  &  Finley  [in 
preparation]  for  details) .  Even  though  Congressionally 
supported,  however,  the  concept  of  structured  training  in  SIMNET 
was  not  immediately  popular.  The  SIMNET  had  already  been  in  use 
for  a  while  and  had  been  used  largely  in  a  free-play  mode.  In 
addition,  as  noted  by  some  key  developers,  training  participants 
and  0/Cs  were  accustomed  to  making  the  decisions  in  managing  and 
controlling  the  training  exercises  themselves.  Unfortunately 
for  those  soldiers  enjoying  free-play  and  accustomed  to 
personally  making  the  training  decisions,  it  was  determined  that 
these  actions  were  counterproductive  to  realizing  SIMNET 's 
training  benefits  (this  was  discussed  earlier  in  the  Background 
section) .  Attempting,  therefore,  to  apply  the  SST  concept  and 
procedures  to  SIMNET  required  some  adjustment  on  the  part  of 
many.  Conducting  exercises  in  accordance  with  standardized  TSPs 
developed  by  others  was  a  new  way  of  doing  business. 

Making  training  decisions.  One  key  developer  commented 
that,  "In  part,  there  is  a  tendency  to  think  that,  because  they 
can  operate  a  piece  of  equipment,  they  automatically  know  all 
about  tactics  and  doctrine,  how  to  implement  them,  and  how  to 
fight."  An  example  of  this  assumption  is  that,  on  not 
infrequent  occasions  in  the  VTP/STRUCCTT  and  SGT  programs, 
military  personnel  would  object  that  many  of  the  TSPs  were  "too 
easy"  and,  hence,  not  worth  their  time.  Many  key  developers 
noted,  however,  that  observation  of  the  performance  of  said 
units,  while  using  these  "too  easy"  TSPs,  often  indicated 
otherwise.  In  fact,  in  some  cases,  they  appeared  to  be  too 
difficult. 

The  CORs  described  two  factors  relating  to  this  problem. 
First,  it  is  doctrine  (DA,  1988,  1990)  that  commanders  are  the 
ones  responsible  for  specifying  the  training  for  their  units, 
based  on  their  knowledge  of  their  unit's  needs.  This  doctrine 
supports  an  apparent  assumption  that  commanders  "know"  what 
their  unit's  training  needs  are  and  how  to  address  them 
correctly.  This  assumption  works  against  acceptance  of  SST 
TSPs,  which  require  adherence  to  training  procedures  that  are 
already  specified  in  detail  (as  is  necessary  if  the  training  is 
to  be  standardized) . 

The  second  factor  is  that  there  is  currently  no  instruction 
systematically  provided  to  officers  or  Non-Commissioned  Officers 
(NCOS)  (on,  e.g.,  SAT  concepts  or  how  to  give  AARs)  to  provide 
them  with  a  basis  for  meeting  their  training  responsibility.  It 
is  not  lack  of  overall  competence;  but  it  may  be  the  fact  that 
unit  commanders  frequently  lack  an  appropriate  basis  for 
evaluating  their  own  unit's  performance  and  then  defining 
training  objectives  based  on  that  knowledge. 

Long-term  support  needed  for  SST.  Both  CORs  and  60%  (9  out 
of  15)  of  the  contractors  identified  needs  for  sustainment  and 
continuing  development  of  SST  TSPs.  Sustainment  issues  included 
distribution  of  TSPs  to  potential  users  and  maintenance  of  a 
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central  TSP  library.  Another  was  the  need  to  update  current 
TSPs  when  doctrinal  and  other  changes  take  place. 

Continuing  development  of  new  TSPs  was  stressed  by  many  key 
developers  as  needed  to  cover  missions,  terrains,  and  forces 
differing  from  those  already  addressed  in  currently  available 
TSPs.  Some  circumstances  needing  new  TSPs  were  described: 
stability  and  support  operations  (SASO) ;  digital  operations;  and 
terrains  and  forces  encountered  in  Somalia,  Bosnia,  and  Korea. 

Another  type  of  needed  support  identified  was  to  educate 
the  Army  regarding  the  SST  concept  and  its  use.  One  key 
developer  summarized  this  need  by  saying:  "Better  education  of 
the  force  is  needed.  [Currently]  only  trial  units  know  about 
the  programs  and  what  they  will  do  for  them." 

Recommended  Solutions 


Given  the  foregoing  description  of  SST  benefits  and 
problems,  possible  solutions  were  formulated  in  terms  of  an  Army 
SST  Master  Plan  and  an  SST  logistics  system.  Possible 
frameworks  for  these  are  outlined  below.  The  SST  Master  Plan 
and  logistics  system  are  seen  as  courses  of  action  that,  if 
adopted,  may  be  able  to  provide  management,  policies,  and 
resources  essential  for  successful  long-term  SST  implementation 
and  sustainment. 

Proposed  Army  SST  Master  Plan 

Suggested  components  for  the  proposed  Master  Plan  are: 

♦  FEAs 

♦  TSP  development 

♦  implementation 

♦  organization (s) 

Each  of  these  components  are  discussed  below. 

To  the  extent  that  any  of  these  areas  are  addressed,  the 
CORs  strongly  advise  the  early  and  continuing  involvement  of 
both  users  (e.g.,  using  Army  units)  and  implementers  (e.g., 

0/Cs)  of  any  products  (e.g.,  TSPs).  They  stated  that  their 
inputs  can  be  valuable  and  their  later  support  will  be 
essential . 

SST  FEAs,  or  needs  analyses.  FEAs  were  described  earlier 
as  one  of  three  areas  of  SST  needs.  Two  major  purposes  were 
identified  for  the  FEAs:  (1)  to  establish  a  foundation  for  an 

Army  SST  Master  Plan  and  (2)  to  identify  those  tasks 
constituting  the  minimum  essential  set  of  tasks  needing  to  be 
trained.  The  FEAs  needed  for  the  SST  Master  Plan  were  described 
as  ones  providing  definition  of  the  Army's  overall  training 
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requirements  and  approaches  to  meeting  these  requirements . 

Here,  one  intent  of  FEAs  might  be  to  fit  SST  into  the  larger^ 
framework  of  Training  Aids,  Devices,  Simulators,  and  Simulations 
(TADSS)  for  the  Combined  Arms  Training  Strategy  (CATS)  (DA, 

1993)  .  This  will  be  addressed  shortly  in  more  detail  as  it 
concerns  SST  implementation. 

Although  discussed  earlier  as  a  need  to  "establish  a 
foundation*  for  a  master  plan,  perhaps  a  statement  of  need  for 
FEAs  should  be  made  a  part  of  the  master  plan  as  well.  Here, 
the  intent  to  identify  requirements  for  and  then  perform  FEAs  as 
needed  could  be  expressed  as  a  part  of  the  plan. 

SST  TSP  development.  As  a  part  of  the  SST  Master  Plan,  a 
statement  of  the  purpose(s),  use,  and  contents  of  SST  TSPs  could 
be  provided.  If  appropriate,  this  could  be  coordinated  with 
updates  to  the  section  in  TRADOC  Regulation  350-70  (DA,  1995) 
covering  TSPs  as  a  general  concept. 

A  section  in  the  proposed  SST  Master  Plan  on  SST 
development  could  include  a  statement  regarding  possible  needs 
for  additional  SST  TSPs  and  examples  of  the  benefits  they  could 
provide.  As  an  example,  a  general  statement  of  the  need  to 
include  CS  and  CSS  positions  and  operations  in  at  least  some 
staff  TSPs,  and  where  this  might  be  most  appropriate,  might  be 
included.  In  addition,  conduct  of  FEAs  could  be  suggested  as  a 
means  of  providing  definitive  information  for  frameworks 
initiating  new  TSP  development  efforts. 

Another  development  consideration  that  may  warrant  attention 
would  be  tools  like  the  Commanders'  Integrated  Training  Tool 
(CITT)  to  support  unit  efforts  to  adapt  existing  SST  TSPs  to 
meet  their  own  individual  requirements  and  constraints.  The 
CITT  is  being  developed  to  support  CCTT,  but  the  concept  should 
be  generally  applicable  to  other  training  simulation  systems  as 
well  (Gossman  et  al.,  1999). 

SST  implementation.  TRADOC 's  CATS  (DA,  1993)  may  need  to 
be  directly  addressed  in  an  SST  Master  Plan.  A  major  focus  of 
CATS  is  on  TADSS  and  planning  for  their  use  in  unit  training _ 
(see  U.S.  Armor  Center,  1999  for  an  example  of  a  branch-specific 
training  strategy  based  on  a  CATS  foundation) .  Simulator  and 
simulation  components  of  TADSS  offering  SST  TSPs  should  merit 
special  attention.  The  concept  of  CITT,  mentioned  above,  could 
be  addressed  in  this  section  of  the  master  plan  as  well.  The 
implementation  concept  could  also  integrate  the  suggestion  that 
SST  should  be  required  of  units  on  a  regular  basis,  being 
scheduled  and  prepared  for  accordingly.  This  is  further 
discussed  below  as  a  part  of  an  SST  logistics  system. 

SST  organization (s) .  The  Army  already  has  a  National 
Simulation  Center  (NSC),  located  at  Fort  Leavenworth,  KS.  Its 
responsibilities  are  largely  concerned  with  simulations  for 
brigade  and  above  (e.g.,  BBS,  Corps  Battle  Simulation  [CBS],  and 
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the  new  Warfighter  Simulation  2000  [WARSIM  2000]).  This  agency 
has  some  simulation  management  and  training  oversight 
responsibilities,  and  is  a  central  facility  for  local  and  off¬ 
site  operations  of  the  CBS.  A  "CTC  for  SST"  idea  could  be 
couched  within  the  framework  of  a  NSC-  and/or  NTC-type  of 
facility /agency  for  collective  training  simulation  devices,  but 
as  one,  instead,  having  responsibilities  for  brigade  and  below 
training.  Training  simulations  and  simulators  of  interest  would 
include  BBS  and  CCTT.  If  the  proposed  concept  of  integrating 
lower  echelon  training  into  WARSIM  2000  is  realized  then  perhaps 
this  could  be  a  part  as  well.  Uncertainty  was  expressed  by  the 
CORs  as  to  whether  this  SST  organizational  concept  could  or 
should  be  realized  as  a  central  facility  or,  rather,  as  a 
geographically  distributed  organization. 

Proposed  SST  Logistics  System 

One  of  the  CORs  expressed  the  need  for  a  means  to  sustain 
the  concept  of  SST  and  the  SST  TSPs  themselves  in  an  especially 
succinct  manner:  "In  the  SST  programs  to  date,  we  have 
developed  a  Development  Focus  Model .  What  we  now  need  is  an 
Implementation  Focus  Model."  The  SST  logistics  system  proposed 
here  may  be  described  as  a  formulation  of  such  a  model .  For 
purposes  of  discussion,  the  label  will  be,  "SST  logistics 
system, "  as  if  it  were  to  be  a  singular  entity.  It  is 
recognized  that  this  capability  could  well  be  an  added 
responsibility  for  already  existing  parties. 

Both  CORs  discussed  the  need  to  determine  how  to  best 
sustain  and  update  the  structured  training  programs,  and  to  then 
implement  these  approaches.  It  was  suggested  that  an  SST 
program  implementation  model  would  be  quite  separate  from  the 
already  established  SST  development  model,  even  though 
complementary.  The  CORs  offered  ideas  regarding  purpose, 
participants,  and  functions  that  a  logistics  system  should 
either  perform  or  be  associated  with,  and  an  operating 
principle . 

Some  logistics  implications  for  training  systems  were 
described  in  addressing  the  implementation  of  embedded  training 
(Cherry,  Peckham,  Purifoy,  &  Roth,  1988) .  This  effort  by  Cherry 
et  al .  is  the  only  one  known  by  ourselves  to  address  a  category 
of  training  systems,  as  opposed  to  one  singular  device.  It  is 
also  the  only  one  we  know  of  which  focuses  attention  on  training 
systems  driven  by  computer  hardware  and  software  requiring 
continuous  monitoring  for  needed  changes  and  timely  responses  to 
meet  these  needs. 

Many  of  the  SST  key  developers  commented  on  the  need  for 
SST  TSPs  to  become  a  regular  part  of  training.  As  this  issue  is 
very  much  a  part  of  the  reason  for  needing  SST  logistical 
support,  this  will  be  discussed  here  before  proceeding  with  the 
topics  of  logistics  purpose,  participants,  and  functions.  The 
"be  a  regular  part  of  training"  concern  was  addressed  with 
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regard  to  two  aspects:  training  scheduling  and  instructional 

emphasis.  The  concept  offered  was  that  SST  should  be  required, 
regularly  scheduled,  and  given  a  high  priority.  The  priority 
given  it  could  be  similar  to  that  of  training  at  NTC  by 
brigades . 

Several  key  developers,  both  CORs  and  contractors, 
cautioned  that  the  prioritization  should  be  focused  on 
instructional  needs  -  not  on,  for  example,  rehearsing  for  a 
later  event.  Many  of  the  key  developers  described  occasions 
where  units  attempted  to  modify  the  SST  TSPs  to  match  conditions 
expected  to  be  a  part  of  their  NTC  rotation  and  then  use  these 
TSPs  as  a  means  for  preparing  for  that  NTC  rotation.  It  was 
noted  by  some  that  these  attempts  appeared  to  result  from  the 
importance  attached  to  performance  at  NTC  rotations  and  a 
failure  to  fully  recognize  the  needs  for  instruction  not 
provided  by  rehearsals. 

In  underscoring  the  instructional  emphasis,  COR  key 
developers  also  cautioned  that  regularly  scheduled  SST  training 
should  be  "task-based,*  that  is,  based  on  identified  training 
requirements.  It  should  be  centrally  focused  on  identifying  and 
addressing  the  unit's  current  skill  deficiencies  and  needs  for 
skill  maintenance.  It  should  not  become  what  is  known  as 
"event-based  training."  This  is  a  term  often  used  to  describe 
training  viewed  as  a  scheduled  event  that  must  be  checked  off 
the  list  of  things  that  must  be  done,  rather  than  as  a  means  for 
addressing  training  needs . 

Logistics  system  purpose.  If  an  Army  SST  Master  Plan  is 
developed,  and  if  the  need  for  support  and  sustainment  is 
accepted,  then  the  purposes  and  functions  of  a  logistics-type 
capability  could  be  a  part  of  this  master  plan.  This  could  be  a 
foundation  upon  which  an  SST  logistics  capability  would  be 
built.  A  dictionary  definition  provides  a  framework  for  more 
exact  definitions  of  the  purposes  to  be  served  by  a  SST 
logistics  system:  Logistics  is  the  procurement,  maintenance, 
distribution,  and  replacement  of  personnel  and  materiel  (Lexical 
&  Electronic  Database  Management  &  Staff,  1994) . 

Logistics  system  participants.  A  general  concept  for  SST 
organization (s)  was  discussed  above  as  part  of  the  SST  Master 
Plan.  Simulation  centers  and  regional  training  centers  would 
certainly  need  to  "buy  into"  the  SST  concept  as  being  the 
providers  thereof  to  units  on  a  regularly  scheduled  basis. 

Having  SST  at  these  facilities  would  create  a  requirement  for 
units  to  come  to  them  if  they  did  not  have  adequate  facilities 
at  their  home  station. 

Other  players  certainly  would  include  the  Army  Training 
Support  Center,  TRADOC,  and  schools  such  as  the  Armor  Center  who 
host  and/or  use  simulation  facilities.  Additional  participants 
would  certainly  include  operating  commands  (e.g.,  U.S.  Army 
Forces  Command)  and  units  (e.g..  First  Cavalry  at  Fort  Hood). 
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Among  other  responsibilities  related  to  logistics,  operating 
commands  and  units  may  be  able  to  identify  training  or  logistics 
requirements  from  their  perspectives  for  which  they  can  provide 
personnel  support  (e.g.,  military  role-players  for  SST 
exercises) . 

Logistics  system  functions.  Functions  for  the  logistics 
system  were  identified  earlier,  as  a  part  of  the  discussion  of 
SST  problems,  as  being:  long  term  support  for  sustainment, 
continuing  TSP  development,  and  force  education.  These  three 
functions  do  not  need  to  be  discussed  further  here.  The  CORs, 
however,  suggested  two  additional  sets  of  functions:  (1) 

oversight  and  management;  and  (2)  monitoring  and  evaluation. 

Their  comments  are  addressed  next. 

Examples  of  the  oversight  and  management  functions 
identified  by  the  CORs  were  to: 

♦  Identify  and  initiate  internally-driven  programmatic 
actions  on  SST  TSPs.  Examples  of  "internally-driven 
programmatic"  changes  are  ones  like  correcting  programming 
errors,  adjusting  to  changes  in  system  hardware  or 
external  software,  or  introducing  possible  enhancements. 

♦  Identify  and  initiate  actions  on  TSP  programs  due  to 
external  reasons  like  changes  in  doctrine  or  need  to  use 
an  existing  TSP  in  a  different  terrain  setting. 

♦  Coordinate  and  systematically  schedule  installation  of  new 
and  modified  hardware  and  software. 

♦  Identify  new  TSPs  that  are  needed  and  prioritize  them  for 
development.  It  was  suggested  that  one  or  more  steering 
committees  might  be  formed  to  perform  this  function. 
Whether  such  committees  should  be  permanent  ones  or, 
instead,  be  assembled  from  the  most  appropriate 
organizations  as  the  need  arises,  or  a  combination  of 
both,  was  discussed  but  not  resolved.  What  was  clear  to 
the  respondents,  however,  was  that  the  members  of  such  a 
steering  committee  should  possess  ranks  and  positions  such 
that  their  decisions  would  be  respected  and  accepted  by 
others . 

♦  Determine  the  tasks,  conditions,  and  standards  to  be  met 
by  contract  logistics  support  (CLS)  personnel  for 
simulation  facilities,  and  bring  all  CLS  contracts  into 
alignment  with  these.  A  follow-on  task  is  to  monitor 
contract  performance  in  coordination  with  local  agencies 
to  ensure  effective  compliance.  When  appropriate  and 
feasible,  an  effort  should  be  made  to  include  a  contract 
requirement  for  truly  qualified  position  role-players,  or 
"training  aids,"  as  needed  to  effectively  support  TSP 
exercises . 


40 


♦  Develop,  manage,  and  sustain  a  team  similar  to  the  concept 
of  a  New  Equipment  Training  (NET)  Team,  or  NET  Team,  to 
train  the  trainers  using  SST.  This  would  be  especially 
valuable  where  SST  trainers  are  unit  personnel  who  are  not 
part  of  a  permanently  established  training  group  having 
that  as  their  primary  mission.  This  is  expected  to  be  a 
continuing  need  for  those  simulations  not  having  a  highly 
skilled  permanent  0/C  team.  (The  VTP  program  is  an 
example  of  a  program  having  a  dedicated  0/C  team.  In 
contrast,  CCTT,  which  STRUCCTT  addressed,  will  not.) 

The  second  additional  set  of  functions,  those  suggested  by 
the  CORs,  monitoring  and  evaluation,  concerned  the  need  to 
identify  requirements  for  changes  to  SST  in  meeting  training 
needs  as  a  member  of  the  overall  Army  training  system.  Such 
changes  could  be  ones  in  roles  or  Standard  Operating  Procedures 
(SOPs) ,  or  changes  to  adjust  its  training  utility  and/or 
effectiveness.  This  concern  was  expressed  with  acknowledgment 
that  evaluative  agencies  addressing  training  have  either  largely 
ceased  to  exist  (e.g.,  TRADOC  School  Directorates  of  Evaluation 
and  Standardization)  or  are  no  longer  strongly  active  in  this 
area  (e.g.,  TRADOC  Systems  Analysis  Activity).  The  TRADOC  and 
others  are  very  concerned  about  this  deficiency,  but  do  not 
presently  have  the  resources  needed  to  increase  their  evaluation 
capabilities.  Perhaps  the  best  solution  for  now  is  to  ensure 
that  any  SST  steering  committee  and/or  logistics  system  be 
designed  to  maintain  effective  coordination  and  feedback  loops 
with  simulation  users. 

Operating  principle.  Both  CORs  strongly  stressed  the  need 
to  use  simulations  in  accordance  with  their  simulation  and 
operation  capabilities.  For  want  of  a  better  label,  we  are 
calling  this  the  "operating  principle."  Examples  of  good  and 
poor  uses  of  SIMNET  and  BBS  were  provided  by  the  CORs: 

♦  The  SIMNET  was  designed  for  training  the  tasks  of  mission 
execution  and  is  very  capable  of  simulating  terrain  and 
other  features  of  the  physical  environment.  But,  while 
using  SIMNET  to  train  tasks  performed  during  the  mission 
phases  of  planning  and  preparation  is  feasible,  the 
simulation  capabilities  of  SIMNET  are  largely  ignored 
because  they  are  unnecessary. 

♦  The  BBS  offers  many  simulation  capabilities  appropriate 
for  large  command  post  exercises.  However,  these 
capabilities  are  not  needed  for,  and  are  entirely  too 
costly  and  large  in  scope  for  supporting  small  group  staff 
training  of  limited  scope.  Here,  vignettes  in  an  ordinary 
office  setting  with  supporting  paperwork  may  provide  all 
that  is  needed  to  simulate  a  working  environment. 


41 


SUMMARY  AND  CONCLUSIONS 


At  the  beginning  of  this  report,  it  was  noted  that  the 
findings  derived  from  the  structured  interview  and  questionnaire 
data  clustered  into  three  main  areas.  These  areas  are; 

♦  Contributions  of  the  constructivism  and  behaviorism 
instructional  theories  to  the  SST  programs; 

♦  Needs  for  further  information  and  research;  and 

♦  Planning  and  logistical  requirements  for  integrating  SST 
into  the  Army's  training  system. 

Contributions  of  the  Constructivism  and  Behaviorism 
Instructional  Theories  to  the  SST  Programs 


Similarities  found  across  the  SST  programs  were  the  use  of 
certain  instructional  design  and  implementation  elements  of 
behaviorism.  These  shared  elements  were:  early  determination 
of  training  objectives  by  the  TSP  design  team;  use  of 
standardized  materials;  TSP  designs  incorporating  a  deliberate 
sequence  of  events  and  being  developed  for  more  novice 
participants;  and  the  provision  of  feedback  by  instructor 
personnel . 

Differences  between  the  SST  programs  were  differences  in 
choice  between  elements  of  behaviorism  and  constructivism.  The 
differences  in  emphasis  were:  procedural  skills  versus 
cognitive  skills;  mastery  versus  experiential  learning;  feedback 
focused  on  actions  versus  processes;  and  linear  versus  non¬ 
linear  TSP  development  processes.  Both  constructivism  and 
behaviorism  elements  were  used,  but  behaviorism  was  used  to  a 
greater  extent  overall .  As  suggested  by  Shlechter  and  Finley 
(in  preparation),  use  of  constructivism  was  greatest  in  COBRAS 
where  it  nearly  equaled  use  of  the  behaviorism  approach. 

Two  questions  were  suggested  earlier  in  this  report: 

♦  Were  the  SST  TSP  instructional  design  elements 
intentionally  selected  from  both  the  constructivism  and 
behaviorism/ SAT  frameworks  based  on  a  comparison  of  their 
elements? 

♦  Did  the  actual  use  of  instructional  design  elements  really 
reflect  both  instructional  theories,  whether  intentional 
or  not? 

Based  on  discussions  with  the  key  developers,  the  answer  to 
the  first  question  is,  "No."  All  of  the  persons  interviewed 
stated  that  they  had  used,  as  their  foundation,  the 
behaviorism/ SAT  procedures  in  TRADOC  Regulation  350-70  (DA, 

1995)  .  The  constructivism  approach  was  not  consciously 
addressed  as  such. 
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The  design  elements  actually  used,  however,  based  again  on 
discussions  with  key  developers,  were  based  on  their 
considerable  experience  combined  with  a  very  close  examination 
of  the  training  requirements  and  their  operational  contexts 
(e.g.,  brigade-level  staff  planning  tasks  as  opposed  to  platoon- 
level  combat  execution  tasks) .  Changes  described  in  SST  reports 
to  the  TSP  design  and  development  procedures  (e.g.,  Campbell  & 
Deter,  1997;  Ford  &  Campbell,  1997;  Shlechter  &  Finley  [in 
preparation]  )  were  the  results  of  these  examinations .  The 
answer  to  the  second  question,  then,  is,  "Yes." 

Overall,  it  appears  that  ARI's  SST  R&D  programs  demonstrate 
the  approach  of  modifying  instructional  design  procedures  based 
on  a  close  examination  of  the  training  requirements  and 
operational  contexts  if  prescribed  design  procedures  do  not  seem 
entirely  adequate  or  appropriate  for  addressing  the 
requirements.  The  prescribed  procedures  were,  in  this  case, 
those  of  behaviorism/SAT  as  covered  in  TRADOC  Regulation  350-70 
(DA,  1995) .  The  need  to  modify  or  enhance  these  procedures 
became  increasingly  evident  as  attention  began  to  focus  more  on 
the  question:  What  are  the  command  and  staff  tasks,  and  the 
demands  of  that  environment?  These  tasks  and  demands  had  not 
yet  been  well  defined  by  the  Army.  They  were  also  understood  to 
usually  have  a  dominant  cognitive  component.  Constructivism  has 
been  argued  to  be  better  suited  to  tasks  having  larger  cognitive 
components.  The  history  of  behaviorism,  in  contrast,  is  that  it 
was  developed  largely  through  examination  of  tasks  that  were 
more  procedural  and  execution-based  and/or,  whatever  the  nature 
of  the  task,  at  least  these  were  the  components  of  the  task 
receiving  the  most  attention. 

Behaviorism,  and  its  implementation  through  SAT,  has  served 
the  Army  very  well  in  assuring  development  of  effective  training 
programs  for  perhaps  the  majority  of  training  requirements.  The 
ARI's  SST  programs,  however,  were  R&D.  As  such,  behaviorism/SAT 
was  not  assumed  to  necessarily  satisfy  all  aspects  of  the 
training  requirements  being  addressed. 

Rather,  ARI  adopted  a  utilitarian  stance  with  regard  to 
instructional  theory  and  application.  The  ARI  SST  TSP  design 
approach  evolved  as  seemed  best  suited  for  addressing  all 
components  of  specific  training  requirements  -  behavioral  and 
cognitive  -  rather  than  adhering  strictly  to  the  behaviorism/SAT 
approach.  The  result,  from  the  perspective  of  alternative 
instructional  theories,  confirms  the  contention  of  Shlechter  and 
Finley  (in  preparation)  that  the  SST  approaches  can  be  described 
as  hybrid  applications  of  the  constructivism  and  behaviorism 
instructional  theories. 

Needs  for  Further  Information  and  Research 

Interviews  provided  the  information  regarding  information 
and  research  needs .  These  needs  can  be  grouped  into  the  areas 
of:  further  developing  the  SST  development  model;  analyses  to 
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address  information  needs;  and  research  on  training  transfer 
strategies,  methods,  and  techniques. 

SST  Development  Model 

Issues  discussed  in  the  Results  section  covering  the  SST 
development  model  concerned  instructional  design  and 
implementation,  and  matching  training  participant  skill  levels 
to  simulation  capabilities.  In  SST  TSP  development  there  is 
design  of  the  scenario  and  then  its  implementation  through 
simulation  software.  The  design  issue  concerned  the  question 
of:  Where  is  it  best  to  apply  instructional  design  elements 

drawn  from  which  alternative  instructional  theories?  The 
development  issue  concerned  the  question  of:  How  to  gain 
greater  and  more  flexible  control  of  the  TSP  scenarios  as 
implemented  by  simulation  software? 

At  least  partial  answers  to  the  first  question  have  been 
afforded  by  the  findings  regarding  constructivism  and 
behaviorism  described  above,  and  suggested  by  the  work  of  Ross 
and  Pierce  (draft  1998) .  Further  answers  could  probably  be 
obtained  through  comparative  analyses  of  different  instructional 
concepts  and  training  requirement  descriptions.  The  goal  would 
be  to  develop  schema (ta)  regarding  appropriate  design  element 
applications.  Instructional  programs  resulting  from  trial  use 
of  the  schema (ta)  could  then  be  evaluated  for  their 
effectiveness  and  efficiency.  Answers  to  the  second  question, 
may,  upon  investigation,  be  found  to  already  exist.  If  not, 
then  perhaps  software  technology  has  already  or  will  soon  be 
advanced  to  the  point  where  developing  such  control  capabilities 
is  both  technically  and  economically  feasible. 

Addressing  the  issue  of  matching  training  participant  skill 
levels  to  simulation  capabilities  may  fall  into  two  types  of 
efforts:  training  development  and  management  guidelines.  If 

the  answer  to  an  earlier  question,  "To  what  extent  might  more 
extensive  use  of  inexpensive  small  group  exercises .. .prepare 
soldiers  to  then  receive  maximvim  training  benefits  from  a  BBS 
exercise?"  is,  "They  could  help  substantially,"  then  effort 
might  be  well  expended  to  develop  additional  exercises . 

Training  management  guidelines  would  then  be  needed  to  assure 
that  these  exercises  are  used  when  needed  to  sustain  skills  and 
to  develop  "gate  skills." 

Analyses  to  Address  Information  Needs 

The  conduct  of  FEAs,  or  needs  analyses,  was  suggested  by  the 
CORs  to  provide  information  useful  for  large-scale  SST  planning 
and  for  identifying  tasks  constituting  minimum  essential  skill 
sets.  Needs  to  plan  for  the  future  of  SST  as  a  part  of  the  Army 
training  system  led  to  the  recommendation  that  FEAs  be  conducted 
to  provide  a  foundation  for  an  Army  SST  Master  Plan.  This 
master  plan  will  be  discussed  in  more  detail  later. 
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Needs  to  maximize  training  benefits  while  minimizing 
training  costs  led  to  discussions  of  whether  there  may  exist 
some  minimum  sets  of  essential  task  skills  that  can  be  acquired 
and  then  sustained  over  time  that  would  meet  particular 
operational  needs  (e.g.,  such  a  set  of  skills  for  an  armor 
unit) .  This  could  be  considered  an  extension  of  the  "critical 
task"  concept  as  used  in  the  conduct  of  task  training  analyses. 
In  the  latter  case,  the  designation  of  "critical"  is  largely  a 
judgment  call  and  it  is  usually  a  rather  independent  one  for 
each  individual  task.  As  an  alternative  to  that  approach,  FEAs 
were  suggested  to  identify  sets  of  tasks  based  on  an  examination 
of  operational  requirements.  If  such  sets  do  exist  and  if 
recurring  training  focused  largely  on  these  tasks  is  found  to 
meet  operational  needs  under  acceptable  conditions,  then  perhaps 
training  effectiveness  would  be  enhanced  and  training  costs 
reduced.  An  exploration  of  this  concept,  at  least,  appears  to 
be  well  worth  the  effort. 

Research  on  Training  Transfer  Strategies,  Methods,  and 
Techniques 

Enhancing  positive  transfer  of  training  to  meet  operational 
requirements  is  clearly  in  the  interests  of  the  Army  if  it  can 
be  done  within  reasonable  cost  constraints.  On  the  other  hand, 
ensuring  that  there  is  at  least  some  positive  transfer  of 
training  is  not  only  desirable,  it  is  necessary.  Three  areas 
were  described  as  directly  relevant  to  these  training  transfer 
concerns:  the  nesting  concept;  adaptive  thinking;  and  training 

bridges.  The  nesting  concept  is  currently  used  in  the  design  of 
SST  TSPs;  the  question  asked  here  was.  To  what  extent  should 
this  concept  continue  to  be  used  exclusively  or,  rather,  should 
more  variety  be  introduced?  To  the  extent  new  SST  TSPs  are 
developed,  the  answer  to  this  question  will  be  especially 
important . 

The  other  two  areas  appear  to  be  ones  which,  if  found  to  be 
feasible  and  successful,  could  also  be  translated  into  training 
methods  and  techniques .  If  the  translations  could  be 
accomplished  within  reasonable  cost  constraints  then  such 
investments  on  the  part  of  the  Army  may  be  desirable  or 
necessary,  depending  on  whether  the  outcome  would  be  enhancement 
or  assurance  of  positive  transfer. 

A  fourth  area  discussed  under  the  Research  heading  was 
training  measurement.  Good  measurement,  automated  if  possible, 
is  key  to  gaining  information  in  mciny  areas,  of  which  training 
is  just  one.  Any  efforts  resulting  in  measures  and  measurement 
techniques  providing  better  and  more  useful  information  would  be 
well  worth  the  expenditure. 
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Planning  and  Logistical  Requirements  for 
Integrating  SST  into  the  Army's  Training  System 

The  general  needs  for  SST  R&D  and  the  resultant  TSPs  were 
described  in  the  Background  section.  As  a  part  of  the  Results 
section  dealing  with  the  proposed  master  plan,  logistics  system, 
and  policy,  a  more  specific  and  up-to-date  background  was 
provided.  Here,  benefits  to  the  Army  resulting  from  use  of  the 
SST  TSPs  were  described.  Also  described  were  cautions  that 
these  benefits  would  not  continue  without  support  and 
sustainment  of  SST  as  a  part  of  the  Army's  training  system. 
Several  problems  were  presented  that  cannot  be  avoided  or 
rectified  without  such  support  and  sustainment. 

Once  the  U.S.  Army  decides  that  a  focused  effort  of  any 
significant  scope  will  be  made  to  address  any  particular  need, 
various  planning  and  other  documents  are  developed  and  staffed. 
In  the  hope  that  such  a  decision  will  be  made  with  regard  to  SST 
and  its  TSPs,  an  Army  Master  Plan  has  been  proposed  for 
consideration  and  an  outline  of  suggested  contents  presented. 

In  addition,  an  SST  logistics  system  has  been  proposed  as  a 
means  for  providing  the  needed  SST  support  and  sustainment. 
Beginning  with  a  general  definition  of  logistics,  the  system's 
purpose,  participants,  functions,  and  one  operating  principle 
were  outlined  and  discussed. 

The  U.S.  Army  needs  effective  and  efficient  training 
resources.  Simulators  and  simulations  offer  the  potential  for 
enormous  training  benefits.  But,  if  they  are  not  used  properly 
in  an  appropriately  structured  manner,  then  they  can  actually 
become  a  negative  resource  -  not  a  positive  one.  Therefore,  we 
strongly  recommend  that  a  decision  be  made  to  begin  a  focused 
effort  to  support  and  sustain  SST  in  the  U.S.  Army. 
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APPENDIX  A 


Contractor  Questionnaire 


A-1 


QUESTIONNAIRE  FOR  KEY  MEMBERS  OF 
THE  VTP  AND  COBRAS  INSTRUCTIONAL  DESIGN  TEAMS 
PT  Number  60-21 


DATA  REQUIRED  BY  THE  PRIVACY  ACT  OF  1974 


AUTHORITY:  Title  10,  USC,  Sec  2358. 

PURPOSE:  As  you  know,  the  Armored  Forces  Research  Unit  (AFRU)  has  been  deeply 
involved  during  the  last  five  years  in  the  research  and  development  of  structured  simulation- 
based  training  programs.  Since  you  have  had  a  key  role  in  the  development  of  these  programs, 
you  may  then  be  able  to  provide  us  with  information  from  those  provided  in  the  published 
reports  concerning  their  instructional  design  and  the  lessons  learned  from  them 

The  data  collected  from  this  form  are  to  be  used  for  research  purposes  only.  Also,  the  data 
will  not  be  used  to  evaluate  the  effectiveness  of  the  developed  programs  nor  the 
associated  published  reports. 

TIME  INVOLVED:  It  should  take  you  less  than  20  minutes  to  complete  this  questiorinaire.  You 
will  have  the  opportunity  to  elaborate  upon  some  of  your  answers  during  the  interview  phase  of 
this  data-collection  effort,  which  should  take  approximately  40  minutes  to  complete.  Hence,  this 
data  collection  should  take  approximately  an  hour  of  your  time. 

DISCLOSURE:  Your  participation  in  this  research  is  strictly  voluntary.  Individuals  are 
encouraged  to  provide  complete  and  accurate  information  in  the  interests  of  the  research,  but 
there  will  be  no  effect  on  individuals  for  not  providing  all  or  any  part  of  the  information 
requested.  This  page  will  be  removed  from  the  remainder  of  the  questionnaire  before  responses 
are  examined  so  that  your  input  will  not  be  identified  when  the  data  are  analyzed. 
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I.  BACKGROUND  ITEMS. 


1 .  Prior  to  working  on  the  AFRU’s  R  &  D  efforts  to  develop  and  field  structured  simulation- 
based  training  programs,  you  have  been:  (You  may  circle  more  than  one  alternative.) 

a.  a  student/trainee  in  a  military  instructional  program. 

b.  a  member  of  a  military  instructional  team. 

c.  an  instructional  designer. 

d.  a  member  of  a  program’s  evaluation  team. 

e.  a  student/trainee  in  a  civilian  instructional  program. 

f.  something  other  than  the  roles  described  above. 

g.  none  of  the  above  as  this  is  my  first  experience  with  a  structured  training  program. 

2.  How  many  total  years  have  you  been  involved  with  the  ARFU’s  efforts  to  develop  and  field 
structured  simulation-based  training  programs? 

a.  five  or  more  years.  b.  two  -  four  years.  c.  one  year  or  less. 

3.  You  were/are  involved  in  which  of  the  following  efforts?  (You  may  circle  more  than  one 
alternative.) 

a.  SIMUTAATTP  b.  SIMBART  c.  SIMUTA-D 

d.  COBRAS  e.  SGT  f.  STRUCCTT  g.  others _ 

4.  Check  your  activity(ies)  for  each  program.  You  may  check  one  or  more  than  one  of  these 
activities  per  program.  Checking  N/A  means  that  you  did  not  work  on  the  program. 

VTP  SIMBART  SIMUTA-D  COBRAS  SGT  STRUCCTT 

Instructional  designer _  _  _  _  _  _ 

Evaluator  _  _  _  _  _  _ 

Supervisor  _  _  _  _  _  _ 

Other  _  _  _  _  _  _ 

N/A  _  _  _  _  _  _ 

5.  For  each  program,  check  the  blank  which  best  reflects  your  answer  to  the  following  question. 
How  well  did  your  professional  or  educational  experiences  prior  to  working  on  the  AFRU’s 
structured  simulation-based  training  program  prepare  you  for  the  activities  checked  in  item  4? 
Checking  N/A  means  that  you  did  not  work  on  this  program. 

VTP  SIMBART  SIMUTA-D  COBRAS  SGT  STRUCCTT 

Extremely  well  _  _  _  _  _  _ 

Well  _  _  _  _  _  _ 

Poorly  _  _  _  _  _  _ 

Not  at  all  _  _  _  _  _  _ 

N/A  _  _  _  _  _  _ 

6.  For  each  program,  check  the  blank  which  best  reflects  your  answer  to  the  following  question. 
How  similar  are  ARFU’s  instructional  programs  to  those  programs  that  you  either  previously 
worked  on  or  took  as  a  student/trainee?  Checking  N/A  means  that  you  did  not  work  on  this 
program. 

VTP  SIMBART  SIMUTA-D  COBRAS  SGT  STRUCCTT 

Extremely  similar  _  _  _  _  _  _ 

Moderately  similar  _  _  _  _  _  _ 

Moderately  different  _  _  _  _  _  _ 

Extremely  different  _  _  _  _  _  _ 

N/A  _  _  _  _  _  _ 


A-3 


Please  answer  this  questionnaire’s  items  in  relation  to  those  programs  which  have  you  helped 
to  design  or  develop.  Separate  response  areas  are  provided  for  the  VTP  (part  A)  and  COBRAS 
(part  B)  in  each  of  the  following  questionnaire  sections.  Feel  free  to  ask  any  question  which  you 
may  have  concerning  the  materials  presented  in  this  questionnaire. _ 


li.  OVERVIEW  ITEMS.  More  than  one  alternative  may  be  circled  for  each  item. 

A.  The  VTP  SET  of  Instructional  Programs  -  SIMUTA,  SIMUTA-D,  SIMBART,  and 
STRUCCTT. 

1.  These  programs  are  most  suitable  for  training  units  whose  personnel  are  at  a(n): 

a.  basic  level/novice  of  armor  expertise. 

b.  intermediate  level  of  armor  expertise. 

c.  advanced/expert  level  of  armor  expertise. 

2.  The  basis(es)  of  the  programs’  instructional  design  was/were: 

a.  previous  SST  training  programs. 

b.  previous  non-SST  military  training  programs. 

c.  previous  civilian  training  programs. 

d.  military  training  doctrine. 

e.  the  instructional  design  literature. 

f.  factors  other  than  those  cited  in  alternatives  a-g. 

3.  Developing  these  programs  involved  overcoming  difficulties  or  constraints: 

a.  not  foreseen  at  the  program’s  on-set. 

b.  inherent  in  working  with  people  with  diverse  backgrounds. 

c.  associated  with  developing  a  novel  set  of  instructional  materials. 

d.  associated  with  the  military  culture. 

e.  other  difficulties  or  constraints  than  those  delineated  in  Alternatives  a-e. 

4.  The  developmental  team  should  have  spent  more  time  on; 

a.  analyzing  the  pre-training  skills  of  prospective  training  participants. 

b.  determining  the  instructional  media. 

c.  determining  the  performance  objectives. 

d.  discussing  issues  with  the  instructional  personnel. 

e.  discussing  issues  with  prospective  training  participants. 

f.  doing  something  other  than  listed  in  Alternatives  a-e. 

g.  doing  none  of  the  above;  the  right  amount  of  time  was  spent  on  the  different 
developmental  activities. 

5.  Applying  your  own  definition  of  “success”  to  those  VTP  programs  you  have  been  involved  in, 
you  would  put  the  success  rate  of  these  programs  for  accomplishing  their  instructional  goals  at: 

SIMUTA  SIMUTA-D  SIMBART  STRUCCTT 

Greater  than  90%.  _  _  _  _ 

Between  75%  and  89%.  _  _  _  _ 

Between  49%  and  74%.  _  _  _  _ 

Between  48%  and  25%  _  _  _  _ 

Between  24%  and  10%.  _  _  _  _ 

Less  than  10%  _  _  _  _ 

An  undetermined  rate.  _  _  _  _ 
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B.  The  COBRAS  SET  of  Instructional  Programs  -  COBRAS  I  -  III  and  SGT. 

1 .  These  programs  are  most  suitable  for  training  units  whose  personnel  are  at  a(n): 

a.  basic  level/novice  of  armor  expertise. 

b.  intermediate  level  of  armor  expertise. 

c.  advanced/expert  level  of  armor  expertise. 

2.  The  basis(es)  of  the  programs’  instructional  design  was/were: 

a.  previous  SST  training  programs. 

b.  previous  non-SST  military  training  programs. 

c.  previous  civilian  training  programs. 

d.  military  training  doctrine. 

e.  the  instructional  design  literature. 

f.  factors  other  than  those  cited  in  alternatives  a-g. 

3.  Developing  these  programs  involved  overcoming  difficulties  or  constraints: 

a.  not  foreseen  at  the  program’s  on-set. 

b.  inherent  in  working  with  people  with  diverse  backgrounds. 

c.  associated  with  developing  a  novel  set  of  instructional  materials. 

d.  associated  with  the  military  culture. 

e.  other  difficulties  or  constraints  than  those  delineated  in  Alternatives  a-e. 

4.  The  developmental  team  should  have  spent  more  time  on: 

a.  analyzing  the  pre-training  skills  of  prospective  training  participants. 

b.  determining  the  instructional  media. 

c.  determining  the  performance  objectives. 

d.  discussing  issues  with  the  instructional  personnel. 

e.  discussing  issues  with  prospective  training  participants. 

f.  doing  something  other  than  listed  in  Alternatives  a-e. 

g.  doing  none  of  the  above;  the  right  amount  of  time  was  spent  on  the  different 
developmental  activities. 

5.  Appling  your  own  definition  of  “success”  to  those  COBRAS  programs  you  have  been 
involved  in,  you  would  put  the  success  rate  of  these  programs  for  accomplishing  their 
instructional  goals  at: 

COBRAS  I  COBRAS  II  COBRAS  III  SGT 

Greater  than  90%.  _  _  _  _ 

Between  75%  and  89%.  _  _  _  _ 

Between  49%  and  74%.  _  _  _  _ 

Between  48%  and  25%  _  _  _  _ 

Between  24%  and  10%.  _  _  _  _ 

Less  than  10%  _  _  _  _ 

An  undetermined  rate.  _  _  _  _ 
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III.  CONSTRUCTIVISnC  OR  BEHAVIORISTIC/SAT  ISSUES.  Remember,  answer  these 
items  in  relation  to  those  programs  which  you  have  helped  to  design,  develop,  and/or 
implement. 


The  instructional  design  literature  has  been  embroiled  in  a  controversy  concerning 
Constructivism  vs.  SAT/Behaviorism.  Characteristics  of  constructivism  and  the  SAT/behaviorism 
approach  are  listed  below.  Answer  these  items  in  relation  to  those  programs  which  you  have 
helped  to  design,  develop,  and/or  implement.  You  may  find  that  a  particular  element  of 
constructivism  or  behaviorism  applies  to  both  the  VTP  and  COBRAS  programs,  to  one  of  these 
programs,  or  to  neither  of  these  programs.  In  any  case,  select  those  items  which  do  apply  to 
your  program. _ _ _ 


A.  The  VTP  Set  of  Instructional  Programs.  Check  elements  which  represent  a  pervasive 
characteristic  of  the  different  VTP  instructional  programs. 

Elements  of  Constructivism 


Training  objectives  determined  by  the  training  participants. 

Training  objectives  also  emerge  as  training  participants  interact  with  the  training  materials. 

Task(s)  immerse(s)  participants  in  realistic  battlefield  conditions  for  their  echelon. 

Course  materials  focused  on  developing  a  unit’s  higher  order  cognitive  skills  (e.g.,  its 
tactical  decision-making). 

Course  materials  focused  on  helping  participants’  develop  the  skills  necessary  to  fight  in 
new  and  different  battlefield  conditions. 

Instructional  program  need  not  contain  a  standardized  set  of  instructional  materials. 

Instructional  program  does  not  contain  a  particular  instructional  sequence  (e.g.,  "crawl- 
walk-run''). 

Instructional  materials  developed  for  the  more  experienced  or  advanced  training 
participants. 

Experiential  learning  is  more  important  than  mastery  learning. 

Instructional  personnel  should  refrain  from  providing  performance  feedback  to  the 
participants  as  they  are  executing  a  table. 

Student-led  AARs. 

Feedback  geared  more  to  the  unit  processes  (e.g.,  communication  among  tanks) 
associated  with  any  particular  action  (e.g.,  getting  to  the  LD/SP  on  time)  than  to  the  action 
itself. 

A  non-linear  or  spiral  progression  used  in  the  instructional  design  process. 


The  VTP  continued: 


Elements  of  SAT/Behaviorism 


Training  objectives  determined  by  the  instructional  design  team. 

Training  objectives  determined  as  an  initial  part  of  the  design  process. 

Task(s)  does  not/do  not  need  to  immerse  training  participants  in  realistic  battlefield 
conditions  for  their  echelon. 

Course  materials  focused  on  developing  the  unit’s  procedural-level  tactical  skills  (e.g., 
executing  tactical  formations). 

Course  materials  focused  on  helping  participants’  develop  the  skills  necessary  to  fight  in 
battlefield  conditions  which  resemble  the  scenario. 

Instructional  program  must  contain  a  standardized  set  of  instructional  materials. 

Instructional  program  contains  a  particular  instructional  sequence  (e.g.,  “crawl-walk-run"). 

Instructional  materials  developed  for  the  less  experienced  or  novice-level 
participants. 

Mastery  learning  is  more  important  than  experiential  learning. 

Instructional  personnel  should,  if  needed,  provide  performance  feedback  to  the 
participants  as  they  are  executing  a  table. 

Instructor-led  AARs. 

Feedback  geared  more  to  the  unit’s  actions  (e.g.,  getting  to  the  LD/SP  on  time)  than  to 
the  processes  (e.g.,  communication  among  tanks)  associated  with  its  action(s). 

A  linear  or  sequential  progression  in  the  instructional  design  process. 
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B.  The  COBRAS  Set  of  Instructional  Programs.  Check  elements  which  represent  a 
pervasive  characteristic  of  the  different  COBRAS  instructional  programs. 

Elements  of  Constructivism 


Training  objectives  determined  by  the  training  participants. 

Training  objectives  also  emerge  as  training  participants  interact  with  the  training  materials. 

Task(s)  immerse(s)  participants  in  realistic  battlefield  conditions  for  their  echelon. 

Course  materials  focused  on  developing  a  unit’s  higher  order  cognitive  skills  (e.g.,  its 
tactical  decision-making). 

Course  materials  focused  on  helping  participants’  develop  the  skills  necessary  to  fight  in 
new  and  different  battlefield  conditions. 

Instructional  program  need  not  contain  a  standardized  set  of  instructional  materials. 

Instructional  program  does  not  contain  a  particular  instructional  sequence  (e.g.,  "crawl- 
walk-run"). 

Instructional  materials  developed  for  the  more  experienced  or  advanced  training 
participants. 

Experiential  learning  is  more  important  than  mastery  learning. 

Instructional  personnel  should  refrain  from  providing  performance  feedback  to  the 
participants  as  they  are  executing  a  table. 

Student-led  AARs. 

Feedback  geared  more  to  the  unit  processes  (e.g.,  communication  among  tanks) 
associated  with  any  particular  action  (e.g.,  getting  to  the  LD/SP  on  time)  than  to  the  action 
itself. 

A  non-linear  or  spiral  progression  used  in  the  instructional  design  process. 


Elements  of  SAT/Behaviorism 

Training  objectives  determined  by  the  instructional  design  team. 

Training  objectives  determined  as  an  initial  part  of  the  design  process. 

Task(s)  does  not/do  not  need  to  immerse  training  participants  in  realistic  battlefield 
conditions  for  their  echelon. 

Course  materials  focused  on  developing  the  unit’s  procedural-level  tactical  skills  (e.g., 
executing  tactical  formations). 

Course  materials  focused  on  helping  participants’  develop  the  skills  necessary  to  fight  in 
battlefield  conditions  which  resemble  the  scenario. 
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B.  COBRAS  continued: 


Instructional  program  must  contain  a  standardized  set  of  instructional  materials. 

Instructional  program  contains  a  particular  instructional  sequence  (e.g.,  "crawl-walk-run"). 

Instructional  materials  developed  for  the  less  experienced  or  novice-level  participants. 

Mastery  learning  is  more  important  than  experiential  learning. 

Instructional  personnel  should,  if  needed,  provide  performance  feedback  to  the 
participants  as  they  are  executing  a  table. 

Instructor-led  AARs. 

Feedback  geared  more  to  the  unit’s  actions  (e.g.,  getting  to  the  LD/SP  on  time)  than  to 
the  processes  (e.g.,  communication  among  tanks)  associated  with  its  action(s). 

A  linear  or  sequential  progression  in  the  instructional  design  process. 


APPENDIX  B 


COR  Structured  Interview  Forms 
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COR  Structured  Interview  Form 


1.  How  did  your  professional  or  educational  experience (s) 
prepare  you  for  the  described  roles? 


We  are  now  going  to  ask  you  some  questions  concerning  the 

(development  or  inqplementation)  of  the  cited  structured  training 

programs . 

2.  As  you  see  it,  what  are  the  two  or  three  most  important 
instructional  goals  of  these  structured  training  programs? 

3.  Do  you  believe  that  these  goals  have  been  realized?  If  so, 
then  how?  If  not,  then  why  not? 

4.  As  you  see  it,  what  are  the  three  most  problematic  aspects 
of  the  structured  training  program?  State  a  reason  for  each 
answer . 

5.  How  was/were  the  structured  training  programs  similar  to 
those  that  you  have  previously  been  associated  with  as 
an  instructional  designer,  evaluator,  instructor  and/or 
trainee? 

6.  How  was/were  the  structured  training  programs  similar  to 
those  that  you  have  previously  been  associated  with  as 
an  instructional  designer,  evaluator,  instructor  and/or 
trainee? 

7.  What  is/are  the  basis  (es)  for  your  answers  to  the  last 
two  questions?  (such  as  previous  experience  with  the  SAT 
processes  or  coursework) 

8.  According  to  your  experience,  what  were  the 

three  most  notable  problems  that  Army  personnel  had  with 
instituting  the  implemented  structured  training  program(s)? 
Also  state  the  reason (s)  for  the  problem. 

9.  What  additional  instructional  or  management  tools  are 
needed  to  help  Army  personnel  sustain  the  implemented 
structured  training  programs? 


The  final  set  of  items  deals  with  an  assortment  of  issues. 

10.  What  are  the  two  or  three  most  salient  contributions  that 
the  structured  training  programs  have  made  to  the  military 
training  community? 

11.  What  is/are  to  you  the  most  gratifying  aspect (s)  of  the 
structured  training  programs? 
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12.  What  two  or  three  things  should  have  been  done  differently 
in  developing  and/or  implementing  these  programs? 

13.  Please  cite  two  or  three  research  and  development  efforts 
that  are  now  needed.  State  a  reason  for  each  need. 
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COR  STRUCTURED  INTERVIEW  QUESTIONS 


♦  How  is  "structured  training"  unique?: 

(1)  Within  the  domain  of  the  instructional  systems 
development  (ISD),  or  systems  approach  to  training  (SAT), 
approaches,  what  is  unique  with  regard  to  "structured  training" 
in  terms  of  either  method  or  product? 

(2)  Which  of  these  "unique"  aspects  do  you  consider  to  be 
truly  unique? 

(3)  Which  of  these  "unique"  aspects  do  you  consider  to 
relate  to  or  derive  from  other  front  end  analysis  techniques 
used  for  other  purposes? 

(4)  Please  describe  each  of  the  latter  with  references  to 
the  extent  possible. 


♦  The  world  of  training  research  and  development  has  moved  from 
a  behaviorist  viewpoint  which  considers  only  the  stimulus  and 
response  (S-R)  to  cognitive  one  which  includes  the  intervening 
cognition  (S-O-R) .  The  focus  of  your,  ARI's,  work  on  structured 
training  has  similarly  evolved  from  a  strictly  behavioral 
psychology  position  to  one  that  encompasses  cognitive 
psychology.  The  focus  has  also  changed  from  one  that  is 
centered  on  the  platoon  to  one  that  encompasses  several  echelons 
(vertical  focus)  and  combined  arms  (horizontal  focus) ,  including 
staffs.  In  summary,  changes  in  the  focus  of  structured  training 
research  include:  (1)  Moving  from  strictly  behavioral  towards 
inclusion  of  cognitive  considerations,  and  (2)  Moving  from 
single  units  only  towards  inclusion  of  vertical  and  horizontal 
players  and  their  relationships.  Please  discuss  the  bases  of 
these  changes.  Were  they: 

(1)  Intentional  and  based  on  a  knowledge  of  the 
overall  movements  of  psychological  research?  If  yes,  what 
research  articles  and  other  sources  of  information? 

(2)  The  result  of  a  common  Zeitgeist,  the  articulation 
of  which  you  shared  with  others?  Who  were  these  "others"?  If  a 
Zeitgeist,  how  long  have  these  "cognitive  psychology"  views  been 
around  and  how  common  were  they? 

(3)  Primarily  a  response  to  directions  and  requests 
from  customers/sponsors. 


♦  What  further  research  is  needed  on  structured  training  as  it 
is  exemplified  by  the  already  developed  TSPs? 
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♦  How  might  the  concept  of  structured  training  and/or  the 
development  methodology  be  further  developed  and/or  expanded? 


♦  To  what  other  training  needs  might  structured  training  be 
usefully  applied?  How  should  the  development  methodology  be 
modified  to  handle  these  needs? 


♦  Do  any  of  the  needs  for  additional  research  derive  from  or 
relate  to  any  current  movements /evolutions  in: 

(1)  Psychology/ training  research? 

(2)  Military  doctrine? 

( 3 )  Technology? 

Please  discuss. 
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COR  INTERVIEW  ITEMS 


The  first  two  interview  items  deal  with  additional  questions  concerning  the  constructivistic  and 
ISO  aspects  of  these  programs.  Remember  - 

For  Constructivism: _ _ _ 

The  instructional  emphasis  is  on:  (a)  authentic  instructional  conditions;  (b)  instructional 
objectives  emerging  from  decisions  made  by  the  training  participants:  (c)  training  participants’ 
developing  an  understanding  of  the  principles  of  tactics;  and  (d)  student-led  AARs. _ 


For  SAT: _ ^ _ 

The  instructional  emphasis  is  on:  (a)  instructional  objectives  being  determined  before  training  is 
given;  (b)  instructional  objectives  being  fulcra  for  developing  the  instructional  materials  (c) 
training  participants’  being  able  to  execute  tactical  actions,  and  (d)  instructor-led  AARs. _ 


1 .  Did  the  developmental  process  employed  for  the  COBRAS  set  of  programs  (exclude  the 
SGT  set  of  programs)  more  closely  resemble  the  practices  of  constructivism  or  traditional  ISD? 
(You  can  answer  by  saying  neither.)  Provide  a  short  rationale  for  your  answer. 


2.  Did  the  developmental  process  employed  for  the  SGT  set  of  programs  more  closely 
resemble  the  practices  of  constructivism  or  traditional  ISD?  (You  can  answer  by  saying 
neither.)  Provide  a  short  rationale  for  your  answer. 


Now,  aside  from  the  issues  of  constructivism  and  ISD  behaviorism: 

3.  In  comparing  the  COBRAS  and  SGT  sets  of  programs: 

a.  What  are  the  two  or  three  most  striking  similarities  between/among  these  programs? 

b.  What  are  the  two  or  three  most  striking  differences  between/among  these  programs? 

4.  What  to  you,  personally,  are  the  most  gratifying  aspects  of  the  structured  training  programs? 

5.  What  additional  R&D  efforts  are  now  needed  to  improve  these  programs? 
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APPENDIX  C 


Contractor  Structured  Interview  Form 
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INTERVIEW  QUESTIONS  FOR  KEY  MEMBERS  OF 
THE  VTP  AND  COBRAS  INSTRUCTIONAL  DESIGN  TEAMS 
PT  Number  60-21 


Please  answer  these  interview  questions  in  relation  to  those  programs  which  you  have  helped 
to  design,  develop,  and/or  implement.  You  were/are  involved  in:  SIMUTA/VTP,  SIMBART, 
SIMUTA-D,  COBRAS.  SGT,  STRUCCTT,  and/or  others? 


A.  Follow-up  questions  on  selected  questionnaire  items.  Ask  the  participants  to  provide 
short  explanations  for  their  responses  to  those  questionnaire  items  selected  by  the 
interviewers. 


B.  Additional  questions. 

1 .  (For  those  who  have  been  associated  with  VTP  and  COBRAS  and  STRUCCTT.)  What 
are  the  two  or  three  most  striking  similarities  between/among  the  VTP,  COBRAS,  and/or 
STRUCCTT  instructional  programs? 


2.  (For  those  who  have  been  associated  with  VTP  and  COBRAS  and  STRUCCTT.)  What 
are  the  two  or  three  most  striking  differences  between/among  the  VTP,  COBRAS,  and/or 
STRUCCTT  instructional  programs? 


3.  What  were/are  the  two  or  three  most  important  instructional  goals  of  these  structured  training 
programs? 
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4.  What  are  the  two  or  three  most  salient  “lessons  learned"  from  these  structured  training 
programs? 


5.  What  are  the  two  or  three  most  salient  contributions  that  the  structured  training  programs 
have  made  to  the  military  training  community? 


6.  What  is/are  the  most  gratifying  aspect(s)  of  the  structured  training  programs? 


7.  What  else  should  have  been  done  in  the  development  and  implementation  of  these 
instructional  programs? 


8.  What  additional  R  &  D  efforts  are  now  needed  to  improve  this  (these)  programs? 
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APPENDIX  D 


Detailed  Breakdown  of  Instructional  Element  Selections 
Made  by  the  Differentiators 
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Table  D-1  describes  the  background  characteristics  of  two 
groups,  the  differentiators  and  the  non-differentiators.  Those 
who  made  differential  selections  of  instructional  design 
elements  between  the  programs  they  had  worked  on  will  be  called 
the  "differentiators."  Those  who  did  not  will  be  called  the 
"non-differentiators."  Table  F-2  will  repeat  the  detailed 
breakouts  and  sums  regarding  selections  of  instructional  design 
elements  found  earlier  in  Table  4  -  but  will  display  only  the 
responses  of  the  six  respondents  out  of  the  11  who  worked  on 
more  than  one  program  and  were  differentiators  in  their 
selections  as  well. 

Backgrounds  of  the  Differentiators  and  the  Non-Differentiators 


The  background  data  from  the  questionnaires  were  examined 
in  a  search  for  items  where  the  frequency  of  responses  was 
distinctly  different  for  the  differentiators  when  compared  to 
the  non-differentiators.  Data  for  three  items  showing  such  a 
difference  are  presented  in  Table  D-1.  While  the  first  item  in 
the  table,  "Studt/ trainee  in  a  military  instruct  prog, "  met  the 
criterion  of  difference,  we  do  not  know  if  this  has  any 
explanation  useful  for  our  question  here.  The  other  two  items, 
however,  "instructional  designer"  and  "evaluator"  do  appear 
useful.  That  is,  it  appears  reasonable  to  suggest  that  having 
even  more  prior  and  SST  program  experience  in  these  two  areas, 
design  and  evaluation,  might  make  the  respondent  more 
knowledgeable  about  and  sensitive  to  differences  between 
constructivism  and  behaviorism/ SAT  elements  in  instructional 
design  and  execution  -  whether  or  not  they  are  familiar  with  the 
labels  for  these  concepts. 


Table  D-1.  Differentiators  compared  to  Non-Differentiators  with 
regard  to  selected  prior  training-related  experiences  and 
similar  SST  program  activities. 


FREQUENCIES  AND  PROPORTIONS 

DIFFERENTIATORS 

N  =  6 

NON-DIFFERENTIATORS 

N  =  5 

EXPERIENCE/ 

ACTIVITIES 

PRIOR 

EXPERIENCE 

SST 

ACTIVITIES 

PRIOR 

EXPERIENCE 

SST 

ACTIVITIES 

Studt /trainee 
in  a  military 
instruct  prog 

2  (.33) 

- 

4  (.80) 

- 

Instructional 

designer 

5  (.83) 

6  (1.00) 

3  (.60) 

4  (.80) 

Evaluator 

4  (.67) 

5  (.83) 

2  (.40) 

2  (.40) 

D-2 


Selections  Made  by  the  Differentiators 


Table  D-2  will  display  the  responses  of  only  the  six 
respondents  out  of  the  11  who  worked  on  more  than  one  program 
and  were  also  differentiators  in  their  selection  of 
constructivism  and  behaviorism/ SAT  elements  for  the  two  or  three 
program  sets  on  which  they  had  worked.  The  respondent  Ns  are  in 
every  case  quite  small  (Ns  =6,  4,  and  2)  .  Despite  this,  the 
data  were  judged  to  be  of  interest  because;  (1)  the  indication 
from  the  background  data  (Table  D-1)  that  these  persons  may  have 
been  differentiating  between  program  sets  on  the  basis  of 
greater  knowledge  and/or  experience  with  instructional  design^ 
and  evaluation,  and  (2)  any  differences  from  and/or  similarities 
with  the  data  presented  for  all  15  respondents  (Tables  4)  may 
serve  to  either  substantiate  the  findings  provided  by  those  data 
or,  instead,  call  them  into  question.  It  can  be  stated  at  the 
outset  that  the  data  presented  in  Table  9  serves  to  substantiate 
the  earlier  findings.  Overall,  the  findings  are  the  same. 

Where  there  are  differences,  they  are  ones  that  accentuate  the 
earlier  findings  in  the  anticipated  directions. 

Table  D-2  presents  the  details  and  a  summary  of  the 
differentiators'  responses.  The  accentuation  of  the  differences 
between  the  VTP/STRUCCTT  and  COBRAS  programs  can  be  noted  by 
describing  differences  in  the  proportions  of  constructivism  and 
behaviorism/ SAT  elements  selected  earlier  in  Table  4  as  compared 
to  Table  D-2.  In  Table  4,  the  difference  between  the 
proportions  for  VTP/STRUCCTT  was  .69  -  .41  =  .28.  In  Table  D-2, 
the  difference  is  greater,  .78  -  .35  =  .43.  The  appearance  in 
Table  4  that  behaviorism/ SAT  was  used  to  a  greater  extent  in  the 
VTP/STRUCCTT  is  sustained  in  Table  D-2.  Although  there  is  a 
change  for  COBRAS,  it  is  quite  small  and  given  the  N,  little  can 
be  said  other  than  that  constructivism  again  appeared  to  be 
applied  to  a  greater  extent  in  COBRAS  than  in  VTP/STRUCCTT. 


D-3 


Table  D-2.  Frequencies  and  proportions  of  selections  made  by  the  differentiators  across  the 
VTP/STRUCCTT  (VTP/STR) ,  SGT,  and  COBRAS  sets  of  programs  regarding  each  of  the  constructivisi 
and  SAT/behaviorism  elements. 
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